Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 506 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.6364. Min Act: -0.1700

Data Index: 471869 (Open Web Text)

Max Activating Token Index: 17

Text #1

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5651. Min Act: -0.1700

Data Index: 5597325 (Open Web Text)

Max Activating Token Index: 48

Text #2

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5526. Min Act: -0.1700

Data Index: 799690 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5522. Min Act: -0.1700

Data Index: 8298374 (Open Web Text)

Max Activating Token Index: 19

Text #4

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5132. Min Act: -0.1700

Data Index: 7974453 (Open Web Text)

Max Activating Token Index: 63

Text #5

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5301. Min Act: -0.1700

Data Index: 7140425 (Open Web Text)

Max Activating Token Index: 10

Text #6

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5163. Min Act: -0.1700

Data Index: 4773965 (Open Web Text)

Max Activating Token Index: 16

Text #7

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5107. Min Act: -0.1700

Data Index: 5957261 (Open Web Text)

Max Activating Token Index: 18

Text #8

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5283. Min Act: -0.1700

Data Index: 3650019 (Open Web Text)

Max Activating Token Index: 35

Text #9

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5042. Min Act: -0.1700

Data Index: 3577952 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.5095. Min Act: -0.1700

Data Index: 1296627 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4844. Min Act: -0.1700

Data Index: 6178503 (Open Web Text)

Max Activating Token Index: 18

Text #12

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4823. Min Act: -0.1700

Data Index: 6760459 (Open Web Text)

Max Activating Token Index: 16

Text #13

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4860. Min Act: -0.1700

Data Index: 4442207 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4817. Min Act: -0.1700

Data Index: 6977669 (Open Web Text)

Max Activating Token Index: 12

Text #15

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4836. Min Act: -0.1700

Data Index: 4577415 (Open Web Text)

Max Activating Token Index: 15

Text #16

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4800. Min Act: -0.1700

Data Index: 624515 (Open Web Text)

Max Activating Token Index: 11

Text #17

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4844. Min Act: -0.1700

Data Index: 3355310 (Open Web Text)

Max Activating Token Index: 53

Text #18

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4714. Min Act: -0.1700

Data Index: 695488 (Open Web Text)

Max Activating Token Index: 9

Text #19

Max Range: 4.6364. Min Range: -4.6364

Max Act: 4.4863. Min Act: -0.1700

Data Index: 6994592 (Open Web Text)

Max Activating Token Index: 15