Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 90 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.1286. Min Act: -0.1700

Data Index: 3597668 (Open Web Text)

Max Activating Token Index: 46

Text #1

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.1194. Min Act: -0.1700

Data Index: 1655802 (Open Web Text)

Max Activating Token Index: 316

Text #2

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.0349. Min Act: -0.1700

Data Index: 6548045 (Open Web Text)

Max Activating Token Index: 919

Text #3

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.0231. Min Act: -0.1700

Data Index: 3028995 (Open Web Text)

Max Activating Token Index: 35

Text #4

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.0394. Min Act: -0.1700

Data Index: 3515495 (Open Web Text)

Max Activating Token Index: 73

Text #5

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.0171. Min Act: -0.1700

Data Index: 6250425 (Open Web Text)

Max Activating Token Index: 392

Text #6

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.0033. Min Act: -0.1678

Data Index: 7871108 (Open Web Text)

Max Activating Token Index: 27

Text #7

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.0090. Min Act: -0.1692

Data Index: 3280293 (Open Web Text)

Max Activating Token Index: 910

Text #8

Max Range: 5.1286. Min Range: -5.1286

Max Act: 5.0137. Min Act: -0.1698

Data Index: 8698043 (Open Web Text)

Max Activating Token Index: 323

Text #9

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.9821. Min Act: -0.1699

Data Index: 4442146 (Open Web Text)

Max Activating Token Index: 326

Text #10

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.9444. Min Act: -0.1644

Data Index: 5076876 (Open Web Text)

Max Activating Token Index: 355

Text #11

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.9459. Min Act: -0.1700

Data Index: 5291341 (Open Web Text)

Max Activating Token Index: 740

Text #12

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.9034. Min Act: -0.1699

Data Index: 3342697 (Open Web Text)

Max Activating Token Index: 979

Text #13

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.9246. Min Act: -0.1700

Data Index: 5137358 (Open Web Text)

Max Activating Token Index: 827

Text #14

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.8857. Min Act: -0.1689

Data Index: 7298248 (Open Web Text)

Max Activating Token Index: 61

Text #15

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.8319. Min Act: -0.1700

Data Index: 335235 (Open Web Text)

Max Activating Token Index: 282

Text #16

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.8641. Min Act: -0.1700

Data Index: 751085 (Open Web Text)

Max Activating Token Index: 728

Text #17

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.8696. Min Act: -0.1700

Data Index: 2358662 (Open Web Text)

Max Activating Token Index: 413

Text #18

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.8684. Min Act: -0.1700

Data Index: 3280291 (Open Web Text)

Max Activating Token Index: 947

Text #19

Max Range: 5.1286. Min Range: -5.1286

Max Act: 4.8584. Min Act: -0.1700

Data Index: 4659188 (Open Web Text)

Max Activating Token Index: 252