Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 303 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3681. Min Act: -0.1700

Data Index: 4933479 (Open Web Text)

Max Activating Token Index: 17

Text #1

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.3184. Min Act: -0.1700

Data Index: 5875717 (Open Web Text)

Max Activating Token Index: 435

Text #2

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2792. Min Act: -0.1700

Data Index: 8458152 (Open Web Text)

Max Activating Token Index: 457

Text #3

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2474. Min Act: -0.1700

Data Index: 5357810 (Open Web Text)

Max Activating Token Index: 442

Text #4

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2784. Min Act: -0.1700

Data Index: 4524669 (Open Web Text)

Max Activating Token Index: 620

Text #5

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2410. Min Act: -0.1700

Data Index: 8458150 (Open Web Text)

Max Activating Token Index: 895

Text #6

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2651. Min Act: -0.1700

Data Index: 8458151 (Open Web Text)

Max Activating Token Index: 464

Text #7

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2599. Min Act: -0.1700

Data Index: 438344 (Open Web Text)

Max Activating Token Index: 45

Text #8

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2247. Min Act: -0.1700

Data Index: 2669009 (Open Web Text)

Max Activating Token Index: 66

Text #9

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2575. Min Act: -0.1700

Data Index: 7446450 (Open Web Text)

Max Activating Token Index: 95

Text #10

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2104. Min Act: -0.1700

Data Index: 5875718 (Open Web Text)

Max Activating Token Index: 153

Text #11

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2345. Min Act: -0.1700

Data Index: 5875715 (Open Web Text)

Max Activating Token Index: 807

Text #12

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2577. Min Act: -0.1700

Data Index: 6835868 (Open Web Text)

Max Activating Token Index: 103

Text #13

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2322. Min Act: -0.1700

Data Index: 4739361 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2119. Min Act: -0.1700

Data Index: 3709810 (Open Web Text)

Max Activating Token Index: 690

Text #15

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2589. Min Act: -0.1700

Data Index: 6835867 (Open Web Text)

Max Activating Token Index: 916

Text #16

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2145. Min Act: -0.1700

Data Index: 7353269 (Open Web Text)

Max Activating Token Index: 205

Text #17

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2197. Min Act: -0.1700

Data Index: 1855571 (Open Web Text)

Max Activating Token Index: 532

Text #18

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2205. Min Act: -0.1700

Data Index: 3461225 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 3.3681. Min Range: -3.3681

Max Act: 3.2040. Min Act: -0.1700

Data Index: 5252712 (Open Web Text)

Max Activating Token Index: 808