Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2313 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.3369. Min Act: -0.1700

Data Index: 7207243 (Open Web Text)

Max Activating Token Index: 120

Text #1

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.3313. Min Act: -0.1700

Data Index: 6889484 (Open Web Text)

Max Activating Token Index: 41

Text #2

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2996. Min Act: -0.1700

Data Index: 3329109 (Open Web Text)

Max Activating Token Index: 834

Text #3

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2766. Min Act: -0.1700

Data Index: 3928373 (Open Web Text)

Max Activating Token Index: 332

Text #4

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2903. Min Act: -0.1700

Data Index: 2949565 (Open Web Text)

Max Activating Token Index: 897

Text #5

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2749. Min Act: -0.1700

Data Index: 3391635 (Open Web Text)

Max Activating Token Index: 968

Text #6

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2790. Min Act: -0.1700

Data Index: 2840913 (Open Web Text)

Max Activating Token Index: 355

Text #7

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2544. Min Act: -0.1700

Data Index: 3418884 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2569. Min Act: -0.1700

Data Index: 801750 (Open Web Text)

Max Activating Token Index: 957

Text #9

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2576. Min Act: -0.1700

Data Index: 5287457 (Open Web Text)

Max Activating Token Index: 180

Text #10

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2573. Min Act: -0.1700

Data Index: 3703076 (Open Web Text)

Max Activating Token Index: 243

Text #11

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2414. Min Act: -0.1700

Data Index: 5647951 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2303. Min Act: -0.1700

Data Index: 944098 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2273. Min Act: -0.1700

Data Index: 3700352 (Open Web Text)

Max Activating Token Index: 591

Text #14

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2180. Min Act: -0.1700

Data Index: 1513087 (Open Web Text)

Max Activating Token Index: 365

Text #15

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2255. Min Act: -0.1700

Data Index: 751726 (Open Web Text)

Max Activating Token Index: 485

Text #16

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2052. Min Act: -0.1700

Data Index: 3588961 (Open Web Text)

Max Activating Token Index: 517

Text #17

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2053. Min Act: -0.1700

Data Index: 3741220 (Open Web Text)

Max Activating Token Index: 347

Text #18

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2036. Min Act: -0.1700

Data Index: 3894830 (Open Web Text)

Max Activating Token Index: 10

Text #19

Max Range: 3.3369. Min Range: -3.3369

Max Act: 3.2106. Min Act: -0.1700

Data Index: 632635 (Open Web Text)

Max Activating Token Index: 13