Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 354 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.9876. Min Act: -0.1700

Data Index: 4120178 (Open Web Text)

Max Activating Token Index: 665

Text #1

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.9361. Min Act: -0.1700

Data Index: 6701861 (Open Web Text)

Max Activating Token Index: 522

Text #2

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.9410. Min Act: -0.1700

Data Index: 7215102 (Open Web Text)

Max Activating Token Index: 147

Text #3

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.9050. Min Act: -0.1700

Data Index: 5844069 (Open Web Text)

Max Activating Token Index: 20

Text #4

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.9094. Min Act: -0.1700

Data Index: 8686592 (Open Web Text)

Max Activating Token Index: 170

Text #5

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8929. Min Act: -0.1700

Data Index: 3580711 (Open Web Text)

Max Activating Token Index: 245

Text #6

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8624. Min Act: -0.1700

Data Index: 2022437 (Open Web Text)

Max Activating Token Index: 521

Text #7

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8924. Min Act: -0.1700

Data Index: 8774335 (Open Web Text)

Max Activating Token Index: 619

Text #8

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8395. Min Act: -0.1700

Data Index: 3140349 (Open Web Text)

Max Activating Token Index: 875

Text #9

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8294. Min Act: -0.1700

Data Index: 6032468 (Open Web Text)

Max Activating Token Index: 739

Text #10

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8174. Min Act: -0.1700

Data Index: 4956261 (Open Web Text)

Max Activating Token Index: 567

Text #11

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8297. Min Act: -0.1700

Data Index: 662027 (Open Web Text)

Max Activating Token Index: 469

Text #12

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8241. Min Act: -0.1700

Data Index: 8219244 (Open Web Text)

Max Activating Token Index: 410

Text #13

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8249. Min Act: -0.1700

Data Index: 4545479 (Open Web Text)

Max Activating Token Index: 627

Text #14

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8161. Min Act: -0.1700

Data Index: 7365328 (Open Web Text)

Max Activating Token Index: 694

Text #15

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8203. Min Act: -0.1700

Data Index: 5241492 (Open Web Text)

Max Activating Token Index: 243

Text #16

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8088. Min Act: -0.1700

Data Index: 6131288 (Open Web Text)

Max Activating Token Index: 784

Text #17

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8096. Min Act: -0.1700

Data Index: 4570314 (Open Web Text)

Max Activating Token Index: 664

Text #18

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.8021. Min Act: -0.1700

Data Index: 6286139 (Open Web Text)

Max Activating Token Index: 537

Text #19

Max Range: 3.9876. Min Range: -3.9876

Max Act: 3.7790. Min Act: -0.1700

Data Index: 2088266 (Open Web Text)

Max Activating Token Index: 958