Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 317 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.1661. Min Act: -0.1700

Data Index: 7149230 (Open Web Text)

Max Activating Token Index: 676

Text #1

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.1574. Min Act: -0.1700

Data Index: 554596 (Open Web Text)

Max Activating Token Index: 814

Text #2

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.1259. Min Act: -0.1700

Data Index: 2080458 (Open Web Text)

Max Activating Token Index: 990

Text #3

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.1089. Min Act: -0.1700

Data Index: 6314658 (Open Web Text)

Max Activating Token Index: 584

Text #4

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.1011. Min Act: -0.1700

Data Index: 5859408 (Open Web Text)

Max Activating Token Index: 632

Text #5

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.0452. Min Act: -0.1700

Data Index: 996957 (Open Web Text)

Max Activating Token Index: 504

Text #6

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.0520. Min Act: -0.1700

Data Index: 4478002 (Open Web Text)

Max Activating Token Index: 515

Text #7

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.0111. Min Act: -0.1700

Data Index: 7734630 (Open Web Text)

Max Activating Token Index: 184

Text #8

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.0325. Min Act: -0.1700

Data Index: 3722727 (Open Web Text)

Max Activating Token Index: 494

Text #9

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.0039. Min Act: -0.1700

Data Index: 7333068 (Open Web Text)

Max Activating Token Index: 218

Text #10

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9949. Min Act: -0.1700

Data Index: 7298683 (Open Web Text)

Max Activating Token Index: 835

Text #11

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9874. Min Act: -0.1700

Data Index: 4108869 (Open Web Text)

Max Activating Token Index: 526

Text #12

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9950. Min Act: -0.1700

Data Index: 3314243 (Open Web Text)

Max Activating Token Index: 1020

Text #13

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9700. Min Act: -0.1700

Data Index: 8789230 (Open Web Text)

Max Activating Token Index: 770

Text #14

Max Range: 3.1661. Min Range: -3.1661

Max Act: 3.0138. Min Act: -0.1700

Data Index: 943313 (Open Web Text)

Max Activating Token Index: 668

Text #15

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9685. Min Act: -0.1700

Data Index: 5611997 (Open Web Text)

Max Activating Token Index: 827

Text #16

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9766. Min Act: -0.1700

Data Index: 4017594 (Open Web Text)

Max Activating Token Index: 324

Text #17

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9331. Min Act: -0.1700

Data Index: 1668815 (Open Web Text)

Max Activating Token Index: 811

Text #18

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9720. Min Act: -0.1700

Data Index: 334542 (Open Web Text)

Max Activating Token Index: 710

Text #19

Max Range: 3.1661. Min Range: -3.1661

Max Act: 2.9493. Min Act: -0.1700

Data Index: 6453411 (Open Web Text)

Max Activating Token Index: 299