Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 540 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9827. Min Act: -0.1700

Data Index: 3825849 (Open Web Text)

Max Activating Token Index: 544

Text #1

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9733. Min Act: -0.1700

Data Index: 5366283 (Open Web Text)

Max Activating Token Index: 477

Text #2

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9829. Min Act: -0.1700

Data Index: 8297720 (Open Web Text)

Max Activating Token Index: 615

Text #3

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9710. Min Act: -0.1700

Data Index: 394124 (Open Web Text)

Max Activating Token Index: 114

Text #4

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9512. Min Act: -0.1700

Data Index: 3903711 (Open Web Text)

Max Activating Token Index: 155

Text #5

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9554. Min Act: -0.1700

Data Index: 5859274 (Open Web Text)

Max Activating Token Index: 823

Text #6

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9396. Min Act: -0.1700

Data Index: 4726889 (Open Web Text)

Max Activating Token Index: 811

Text #7

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9334. Min Act: -0.1700

Data Index: 3422468 (Open Web Text)

Max Activating Token Index: 587

Text #8

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9341. Min Act: -0.1700

Data Index: 7664646 (Open Web Text)

Max Activating Token Index: 958

Text #9

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9122. Min Act: -0.1700

Data Index: 5977720 (Open Web Text)

Max Activating Token Index: 641

Text #10

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9193. Min Act: -0.1700

Data Index: 368643 (Open Web Text)

Max Activating Token Index: 161

Text #11

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9272. Min Act: -0.1700

Data Index: 4392379 (Open Web Text)

Max Activating Token Index: 253

Text #12

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9242. Min Act: -0.1700

Data Index: 36569 (Open Web Text)

Max Activating Token Index: 859

Text #13

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9085. Min Act: -0.1700

Data Index: 5458238 (Open Web Text)

Max Activating Token Index: 674

Text #14

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9125. Min Act: -0.1700

Data Index: 8278741 (Open Web Text)

Max Activating Token Index: 650

Text #15

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9056. Min Act: -0.1700

Data Index: 7837351 (Open Web Text)

Max Activating Token Index: 690

Text #16

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9107. Min Act: -0.1700

Data Index: 3743609 (Open Web Text)

Max Activating Token Index: 521

Text #17

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.8951. Min Act: -0.1700

Data Index: 2927537 (Open Web Text)

Max Activating Token Index: 263

Text #18

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.9003. Min Act: -0.1700

Data Index: 5921957 (Open Web Text)

Max Activating Token Index: 1014

Text #19

Max Range: 1.9829. Min Range: -1.9829

Max Act: 1.8900. Min Act: -0.1700

Data Index: 57711 (Open Web Text)

Max Activating Token Index: 752