Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 351 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.5912. Min Act: -0.1700

Data Index: 4004438 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.5383. Min Act: -0.1700

Data Index: 2740008 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.5372. Min Act: -0.1700

Data Index: 134491 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.5117. Min Act: -0.1700

Data Index: 4326500 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.5040. Min Act: -0.1700

Data Index: 1238093 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4996. Min Act: -0.1700

Data Index: 814073 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4881. Min Act: -0.1700

Data Index: 7283080 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4822. Min Act: -0.1700

Data Index: 1521615 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4731. Min Act: -0.1700

Data Index: 4310736 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4364. Min Act: -0.1700

Data Index: 1597912 (Open Web Text)

Max Activating Token Index: 20

Text #10

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4408. Min Act: -0.1700

Data Index: 6751929 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4408. Min Act: -0.1700

Data Index: 1870263 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4350. Min Act: -0.1700

Data Index: 603079 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4304. Min Act: -0.1700

Data Index: 2000705 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4439. Min Act: -0.1700

Data Index: 8501837 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4258. Min Act: -0.1700

Data Index: 6824615 (Open Web Text)

Max Activating Token Index: 229

Text #16

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4297. Min Act: -0.1700

Data Index: 2689797 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4212. Min Act: -0.1700

Data Index: 4200789 (Open Web Text)

Max Activating Token Index: 16

Text #18

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4282. Min Act: -0.1700

Data Index: 1755632 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 2.5912. Min Range: -2.5912

Max Act: 2.4262. Min Act: -0.1700

Data Index: 2646743 (Open Web Text)

Max Activating Token Index: 3