Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1673 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.3531. Min Act: -0.1700

Data Index: 4724057 (Open Web Text)

Max Activating Token Index: 297

Text #1

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.3042. Min Act: -0.1700

Data Index: 31506 (Open Web Text)

Max Activating Token Index: 253

Text #2

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.2897. Min Act: -0.1700

Data Index: 8097456 (Open Web Text)

Max Activating Token Index: 205

Text #3

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.2562. Min Act: -0.1700

Data Index: 6921812 (Open Web Text)

Max Activating Token Index: 719

Text #4

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.2216. Min Act: -0.1700

Data Index: 7090254 (Open Web Text)

Max Activating Token Index: 436

Text #5

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1949. Min Act: -0.1700

Data Index: 7669988 (Open Web Text)

Max Activating Token Index: 943

Text #6

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.2232. Min Act: -0.1700

Data Index: 4188583 (Open Web Text)

Max Activating Token Index: 852

Text #7

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1651. Min Act: -0.1700

Data Index: 4343 (Open Web Text)

Max Activating Token Index: 295

Text #8

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1650. Min Act: -0.1700

Data Index: 4176336 (Open Web Text)

Max Activating Token Index: 566

Text #9

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1420. Min Act: -0.1700

Data Index: 1797611 (Open Web Text)

Max Activating Token Index: 610

Text #10

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1047. Min Act: -0.1700

Data Index: 7775244 (Open Web Text)

Max Activating Token Index: 1022

Text #11

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1269. Min Act: -0.1700

Data Index: 597245 (Open Web Text)

Max Activating Token Index: 451

Text #12

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.0838. Min Act: -0.1700

Data Index: 1684791 (Open Web Text)

Max Activating Token Index: 201

Text #13

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1283. Min Act: -0.1700

Data Index: 1714913 (Open Web Text)

Max Activating Token Index: 861

Text #14

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1105. Min Act: -0.1700

Data Index: 78647 (Open Web Text)

Max Activating Token Index: 751

Text #15

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.1133. Min Act: -0.1700

Data Index: 7434439 (Open Web Text)

Max Activating Token Index: 201

Text #16

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.0474. Min Act: -0.1700

Data Index: 4924115 (Open Web Text)

Max Activating Token Index: 634

Text #17

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.0527. Min Act: -0.1700

Data Index: 5505103 (Open Web Text)

Max Activating Token Index: 1017

Text #18

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.0440. Min Act: -0.1700

Data Index: 6516228 (Open Web Text)

Max Activating Token Index: 870

Text #19

Max Range: 4.3531. Min Range: -4.3531

Max Act: 4.0415. Min Act: -0.1700

Data Index: 3159807 (Open Web Text)

Max Activating Token Index: 14