Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2332 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.8967. Min Act: -0.1700

Data Index: 215941 (Open Web Text)

Max Activating Token Index: 17

Text #1

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.8307. Min Act: -0.1700

Data Index: 3805594 (Open Web Text)

Max Activating Token Index: 941

Text #2

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.8166. Min Act: -0.1700

Data Index: 3805584 (Open Web Text)

Max Activating Token Index: 32

Text #3

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7989. Min Act: -0.1700

Data Index: 694820 (Open Web Text)

Max Activating Token Index: 940

Text #4

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7841. Min Act: -0.1700

Data Index: 8234950 (Open Web Text)

Max Activating Token Index: 425

Text #5

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7869. Min Act: -0.1447

Data Index: 5613973 (Open Web Text)

Max Activating Token Index: 12

Text #6

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7853. Min Act: -0.1700

Data Index: 3805586 (Open Web Text)

Max Activating Token Index: 203

Text #7

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7700. Min Act: -0.1700

Data Index: 3805614 (Open Web Text)

Max Activating Token Index: 661

Text #8

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7496. Min Act: -0.1700

Data Index: 694814 (Open Web Text)

Max Activating Token Index: 418

Text #9

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7494. Min Act: -0.1700

Data Index: 694800 (Open Web Text)

Max Activating Token Index: 684

Text #10

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7432. Min Act: -0.1700

Data Index: 3824519 (Open Web Text)

Max Activating Token Index: 17

Text #11

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7528. Min Act: -0.1700

Data Index: 694815 (Open Web Text)

Max Activating Token Index: 132

Text #12

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.6831. Min Act: -0.1700

Data Index: 694794 (Open Web Text)

Max Activating Token Index: 181

Text #13

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7409. Min Act: -0.1700

Data Index: 3805603 (Open Web Text)

Max Activating Token Index: 685

Text #14

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7176. Min Act: -0.1700

Data Index: 3805611 (Open Web Text)

Max Activating Token Index: 830

Text #15

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.6998. Min Act: -0.1700

Data Index: 3805597 (Open Web Text)

Max Activating Token Index: 234

Text #16

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7142. Min Act: -0.1700

Data Index: 694796 (Open Web Text)

Max Activating Token Index: 308

Text #17

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7184. Min Act: -0.1700

Data Index: 694813 (Open Web Text)

Max Activating Token Index: 678

Text #18

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7296. Min Act: -0.1699

Data Index: 1151331 (Open Web Text)

Max Activating Token Index: 13

Text #19

Max Range: 3.8967. Min Range: -3.8967

Max Act: 3.7054. Min Act: -0.1700

Data Index: 694808 (Open Web Text)

Max Activating Token Index: 625