Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1725 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2416. Min Act: -0.1700

Data Index: 7908431 (Open Web Text)

Max Activating Token Index: 185

Text #1

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.2171. Min Act: -0.1700

Data Index: 5975990 (Open Web Text)

Max Activating Token Index: 188

Text #2

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1725. Min Act: -0.1700

Data Index: 3127942 (Open Web Text)

Max Activating Token Index: 751

Text #3

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1761. Min Act: -0.1700

Data Index: 6932203 (Open Web Text)

Max Activating Token Index: 187

Text #4

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1668. Min Act: -0.1700

Data Index: 5524300 (Open Web Text)

Max Activating Token Index: 206

Text #5

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1534. Min Act: -0.1700

Data Index: 8203370 (Open Web Text)

Max Activating Token Index: 720

Text #6

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1409. Min Act: -0.1700

Data Index: 8203099 (Open Web Text)

Max Activating Token Index: 743

Text #7

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1358. Min Act: -0.1700

Data Index: 577101 (Open Web Text)

Max Activating Token Index: 816

Text #8

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1404. Min Act: -0.1700

Data Index: 5260539 (Open Web Text)

Max Activating Token Index: 777

Text #9

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1232. Min Act: -0.1700

Data Index: 7054601 (Open Web Text)

Max Activating Token Index: 759

Text #10

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1184. Min Act: -0.1700

Data Index: 2906042 (Open Web Text)

Max Activating Token Index: 769

Text #11

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1236. Min Act: -0.1700

Data Index: 598191 (Open Web Text)

Max Activating Token Index: 765

Text #12

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1205. Min Act: -0.1700

Data Index: 6401214 (Open Web Text)

Max Activating Token Index: 818

Text #13

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.1280. Min Act: -0.1700

Data Index: 1403501 (Open Web Text)

Max Activating Token Index: 743

Text #14

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.0889. Min Act: -0.1700

Data Index: 6759528 (Open Web Text)

Max Activating Token Index: 227

Text #15

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.0500. Min Act: -0.1700

Data Index: 3410847 (Open Web Text)

Max Activating Token Index: 268

Text #16

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.0834. Min Act: -0.1700

Data Index: 1407656 (Open Web Text)

Max Activating Token Index: 779

Text #17

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.0654. Min Act: -0.1700

Data Index: 5678641 (Open Web Text)

Max Activating Token Index: 776

Text #18

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.0748. Min Act: -0.1700

Data Index: 4156225 (Open Web Text)

Max Activating Token Index: 754

Text #19

Max Range: 2.2416. Min Range: -2.2416

Max Act: 2.0747. Min Act: -0.1700

Data Index: 3713669 (Open Web Text)

Max Activating Token Index: 818