Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 824 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.5634. Min Act: -0.1700

Data Index: 7609858 (Open Web Text)

Max Activating Token Index: 828

Text #1

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.3159. Min Act: -0.1700

Data Index: 684498 (Open Web Text)

Max Activating Token Index: 372

Text #2

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.2510. Min Act: -0.1700

Data Index: 2590352 (Open Web Text)

Max Activating Token Index: 222

Text #3

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.1300. Min Act: -0.1700

Data Index: 1578459 (Open Web Text)

Max Activating Token Index: 243

Text #4

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.2118. Min Act: -0.1700

Data Index: 8429357 (Open Web Text)

Max Activating Token Index: 498

Text #5

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.1066. Min Act: -0.1700

Data Index: 5560684 (Open Web Text)

Max Activating Token Index: 513

Text #6

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.0973. Min Act: -0.1700

Data Index: 655608 (Open Web Text)

Max Activating Token Index: 879

Text #7

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.0254. Min Act: -0.1700

Data Index: 840747 (Open Web Text)

Max Activating Token Index: 435

Text #8

Max Range: 9.5634. Min Range: -9.5634

Max Act: 9.0285. Min Act: -0.1700

Data Index: 6969337 (Open Web Text)

Max Activating Token Index: 617

Text #9

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.9475. Min Act: -0.1700

Data Index: 5506052 (Open Web Text)

Max Activating Token Index: 580

Text #10

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.9612. Min Act: -0.1700

Data Index: 5616663 (Open Web Text)

Max Activating Token Index: 438

Text #11

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.9024. Min Act: -0.1700

Data Index: 6252213 (Open Web Text)

Max Activating Token Index: 12

Text #12

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.8983. Min Act: -0.1700

Data Index: 3654113 (Open Web Text)

Max Activating Token Index: 676

Text #13

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.9350. Min Act: -0.1700

Data Index: 8030510 (Open Web Text)

Max Activating Token Index: 183

Text #14

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.8838. Min Act: -0.1700

Data Index: 3279677 (Open Web Text)

Max Activating Token Index: 465

Text #15

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.9107. Min Act: -0.1700

Data Index: 6223111 (Open Web Text)

Max Activating Token Index: 436

Text #16

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.8717. Min Act: -0.1700

Data Index: 3518315 (Open Web Text)

Max Activating Token Index: 668

Text #17

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.8788. Min Act: -0.1700

Data Index: 7093951 (Open Web Text)

Max Activating Token Index: 462

Text #18

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.8052. Min Act: -0.1700

Data Index: 431379 (Open Web Text)

Max Activating Token Index: 92

Text #19

Max Range: 9.5634. Min Range: -9.5634

Max Act: 8.8828. Min Act: -0.1700

Data Index: 5459455 (Open Web Text)

Max Activating Token Index: 567