Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2151 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.7406. Min Act: -0.1700

Data Index: 1812136 (Open Web Text)

Max Activating Token Index: 268

Text #1

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.6697. Min Act: -0.1700

Data Index: 3556050 (Open Web Text)

Max Activating Token Index: 30

Text #2

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5972. Min Act: -0.1700

Data Index: 8223915 (Open Web Text)

Max Activating Token Index: 984

Text #3

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.6210. Min Act: -0.1700

Data Index: 5805866 (Open Web Text)

Max Activating Token Index: 281

Text #4

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5654. Min Act: -0.1700

Data Index: 731270 (Open Web Text)

Max Activating Token Index: 564

Text #5

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5328. Min Act: -0.1700

Data Index: 2796276 (Open Web Text)

Max Activating Token Index: 995

Text #6

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5450. Min Act: -0.1700

Data Index: 7373226 (Open Web Text)

Max Activating Token Index: 974

Text #7

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5573. Min Act: -0.1700

Data Index: 1200242 (Open Web Text)

Max Activating Token Index: 875

Text #8

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5400. Min Act: -0.1700

Data Index: 4352385 (Open Web Text)

Max Activating Token Index: 1008

Text #9

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5119. Min Act: -0.1700

Data Index: 5919946 (Open Web Text)

Max Activating Token Index: 833

Text #10

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.5057. Min Act: -0.1700

Data Index: 8597962 (Open Web Text)

Max Activating Token Index: 822

Text #11

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4988. Min Act: -0.1700

Data Index: 5030706 (Open Web Text)

Max Activating Token Index: 903

Text #12

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4862. Min Act: -0.1700

Data Index: 744689 (Open Web Text)

Max Activating Token Index: 552

Text #13

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4761. Min Act: -0.1700

Data Index: 2779263 (Open Web Text)

Max Activating Token Index: 457

Text #14

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4580. Min Act: -0.1700

Data Index: 760617 (Open Web Text)

Max Activating Token Index: 890

Text #15

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4761. Min Act: -0.1700

Data Index: 8554440 (Open Web Text)

Max Activating Token Index: 859

Text #16

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4857. Min Act: -0.1700

Data Index: 2221496 (Open Web Text)

Max Activating Token Index: 989

Text #17

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4788. Min Act: -0.1700

Data Index: 549732 (Open Web Text)

Max Activating Token Index: 450

Text #18

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4899. Min Act: -0.1700

Data Index: 4955675 (Open Web Text)

Max Activating Token Index: 970

Text #19

Max Range: 3.7406. Min Range: -3.7406

Max Act: 3.4571. Min Act: -0.1700

Data Index: 3087771 (Open Web Text)

Max Activating Token Index: 710