Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2737 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.3433. Min Act: -0.1700

Data Index: 4638653 (Open Web Text)

Max Activating Token Index: 698

Text #1

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.3203. Min Act: -0.1700

Data Index: 4965717 (Open Web Text)

Max Activating Token Index: 17

Text #2

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.2685. Min Act: -0.1700

Data Index: 372515 (Open Web Text)

Max Activating Token Index: 949

Text #3

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1728. Min Act: -0.1700

Data Index: 1682047 (Open Web Text)

Max Activating Token Index: 22

Text #4

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1567. Min Act: -0.1700

Data Index: 6122992 (Open Web Text)

Max Activating Token Index: 573

Text #5

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1586. Min Act: -0.1700

Data Index: 6061432 (Open Web Text)

Max Activating Token Index: 129

Text #6

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1416. Min Act: -0.1700

Data Index: 4947299 (Open Web Text)

Max Activating Token Index: 65

Text #7

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1226. Min Act: -0.1700

Data Index: 5004114 (Open Web Text)

Max Activating Token Index: 713

Text #8

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1146. Min Act: -0.1700

Data Index: 6823672 (Open Web Text)

Max Activating Token Index: 193

Text #9

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1255. Min Act: -0.1700

Data Index: 2853677 (Open Web Text)

Max Activating Token Index: 689

Text #10

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1247. Min Act: -0.1700

Data Index: 7816306 (Open Web Text)

Max Activating Token Index: 203

Text #11

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1262. Min Act: -0.1700

Data Index: 811822 (Open Web Text)

Max Activating Token Index: 635

Text #12

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1541. Min Act: -0.1700

Data Index: 392251 (Open Web Text)

Max Activating Token Index: 104

Text #13

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.0676. Min Act: -0.1700

Data Index: 7670301 (Open Web Text)

Max Activating Token Index: 249

Text #14

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1181. Min Act: -0.1700

Data Index: 1728667 (Open Web Text)

Max Activating Token Index: 353

Text #15

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1063. Min Act: -0.1700

Data Index: 8696483 (Open Web Text)

Max Activating Token Index: 645

Text #16

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.1049. Min Act: -0.1700

Data Index: 1719422 (Open Web Text)

Max Activating Token Index: 327

Text #17

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.0610. Min Act: -0.1700

Data Index: 3467923 (Open Web Text)

Max Activating Token Index: 515

Text #18

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.0732. Min Act: -0.1700

Data Index: 881005 (Open Web Text)

Max Activating Token Index: 8

Text #19

Max Range: 3.3433. Min Range: -3.3433

Max Act: 3.0686. Min Act: -0.1700

Data Index: 5704271 (Open Web Text)

Max Activating Token Index: 991