Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 770 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.9232. Min Act: -0.1700

Data Index: 4753723 (Open Web Text)

Max Activating Token Index: 638

Text #1

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.9159. Min Act: -0.1700

Data Index: 4074273 (Open Web Text)

Max Activating Token Index: 453

Text #2

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.9008. Min Act: -0.1700

Data Index: 5097547 (Open Web Text)

Max Activating Token Index: 168

Text #3

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8622. Min Act: -0.1699

Data Index: 5299634 (Open Web Text)

Max Activating Token Index: 228

Text #4

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8633. Min Act: -0.1700

Data Index: 7667054 (Open Web Text)

Max Activating Token Index: 122

Text #5

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8578. Min Act: -0.1700

Data Index: 1853055 (Open Web Text)

Max Activating Token Index: 859

Text #6

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8574. Min Act: -0.1700

Data Index: 4776543 (Open Web Text)

Max Activating Token Index: 32

Text #7

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8430. Min Act: -0.1700

Data Index: 4435393 (Open Web Text)

Max Activating Token Index: 490

Text #8

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8559. Min Act: -0.1700

Data Index: 2141941 (Open Web Text)

Max Activating Token Index: 34

Text #9

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8475. Min Act: -0.1700

Data Index: 2431714 (Open Web Text)

Max Activating Token Index: 15

Text #10

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8540. Min Act: -0.1698

Data Index: 2306165 (Open Web Text)

Max Activating Token Index: 677

Text #11

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8444. Min Act: -0.1700

Data Index: 8162342 (Open Web Text)

Max Activating Token Index: 507

Text #12

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8336. Min Act: -0.1700

Data Index: 999986 (Open Web Text)

Max Activating Token Index: 451

Text #13

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8355. Min Act: -0.1700

Data Index: 6100691 (Open Web Text)

Max Activating Token Index: 885

Text #14

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8408. Min Act: -0.1700

Data Index: 4982572 (Open Web Text)

Max Activating Token Index: 556

Text #15

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8389. Min Act: -0.1700

Data Index: 7765585 (Open Web Text)

Max Activating Token Index: 334

Text #16

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8255. Min Act: -0.1700

Data Index: 1459740 (Open Web Text)

Max Activating Token Index: 128

Text #17

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8286. Min Act: -0.1700

Data Index: 1875073 (Open Web Text)

Max Activating Token Index: 537

Text #18

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8274. Min Act: -0.1700

Data Index: 236920 (Open Web Text)

Max Activating Token Index: 153

Text #19

Max Range: 1.9232. Min Range: -1.9232

Max Act: 1.8287. Min Act: -0.1700

Data Index: 7902602 (Open Web Text)

Max Activating Token Index: 519