Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2059 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.1052. Min Range: -6.1052

Max Act: 6.1052. Min Act: -0.1700

Data Index: 2155127 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.9722. Min Act: -0.1700

Data Index: 3669473 (Open Web Text)

Max Activating Token Index: 10

Text #2

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8802. Min Act: -0.1700

Data Index: 7601455 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8988. Min Act: -0.1700

Data Index: 3845070 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8726. Min Act: -0.1700

Data Index: 6297372 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8454. Min Act: -0.1700

Data Index: 5034621 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8350. Min Act: -0.1700

Data Index: 5013417 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8385. Min Act: -0.1700

Data Index: 6727182 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8389. Min Act: -0.1700

Data Index: 1988523 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8408. Min Act: -0.1700

Data Index: 2603106 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8497. Min Act: -0.1700

Data Index: 199424 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8289. Min Act: -0.1700

Data Index: 3487400 (Open Web Text)

Max Activating Token Index: 8

Text #12

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8123. Min Act: -0.1700

Data Index: 1004213 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8276. Min Act: -0.1700

Data Index: 6099832 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8146. Min Act: -0.1700

Data Index: 5507433 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.8332. Min Act: -0.1700

Data Index: 7783621 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.7902. Min Act: -0.1700

Data Index: 4148995 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.7729. Min Act: -0.1700

Data Index: 4044891 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.7706. Min Act: -0.1700

Data Index: 593069 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 6.1052. Min Range: -6.1052

Max Act: 5.7784. Min Act: -0.1700

Data Index: 5135825 (Open Web Text)

Max Activating Token Index: 6