Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1154 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.9173. Min Act: -0.1700

Data Index: 1950834 (Open Web Text)

Max Activating Token Index: 113

Text #1

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.8112. Min Act: -0.1700

Data Index: 2169645 (Open Web Text)

Max Activating Token Index: 767

Text #2

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7917. Min Act: -0.1700

Data Index: 7551226 (Open Web Text)

Max Activating Token Index: 751

Text #3

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.8465. Min Act: -0.1700

Data Index: 19523 (Open Web Text)

Max Activating Token Index: 419

Text #4

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7871. Min Act: -0.1700

Data Index: 639757 (Open Web Text)

Max Activating Token Index: 754

Text #5

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7831. Min Act: -0.1700

Data Index: 1831978 (Open Web Text)

Max Activating Token Index: 469

Text #6

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7359. Min Act: -0.1700

Data Index: 5577900 (Open Web Text)

Max Activating Token Index: 930

Text #7

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7792. Min Act: -0.1700

Data Index: 7233762 (Open Web Text)

Max Activating Token Index: 835

Text #8

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7327. Min Act: -0.1700

Data Index: 4151617 (Open Web Text)

Max Activating Token Index: 837

Text #9

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7600. Min Act: -0.1700

Data Index: 4737246 (Open Web Text)

Max Activating Token Index: 654

Text #10

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.6910. Min Act: -0.1700

Data Index: 1682724 (Open Web Text)

Max Activating Token Index: 733

Text #11

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7263. Min Act: -0.1700

Data Index: 2872433 (Open Web Text)

Max Activating Token Index: 952

Text #12

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7292. Min Act: -0.1700

Data Index: 5000136 (Open Web Text)

Max Activating Token Index: 260

Text #13

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7230. Min Act: -0.1700

Data Index: 688674 (Open Web Text)

Max Activating Token Index: 421

Text #14

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7115. Min Act: -0.1700

Data Index: 5891694 (Open Web Text)

Max Activating Token Index: 924

Text #15

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7225. Min Act: -0.1700

Data Index: 8003160 (Open Web Text)

Max Activating Token Index: 279

Text #16

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.6983. Min Act: -0.1700

Data Index: 3128504 (Open Web Text)

Max Activating Token Index: 191

Text #17

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7308. Min Act: -0.1700

Data Index: 5169279 (Open Web Text)

Max Activating Token Index: 327

Text #18

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7182. Min Act: -0.1700

Data Index: 2499247 (Open Web Text)

Max Activating Token Index: 764

Text #19

Max Range: 3.9173. Min Range: -3.9173

Max Act: 3.7001. Min Act: -0.1700

Data Index: 811144 (Open Web Text)

Max Activating Token Index: 160