Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2391 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.9013. Min Act: -0.1700

Data Index: 664641 (Open Web Text)

Max Activating Token Index: 883

Text #1

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.8271. Min Act: -0.1700

Data Index: 4261469 (Open Web Text)

Max Activating Token Index: 202

Text #2

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.8408. Min Act: -0.1700

Data Index: 1289933 (Open Web Text)

Max Activating Token Index: 308

Text #3

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7849. Min Act: -0.1700

Data Index: 2279724 (Open Web Text)

Max Activating Token Index: 817

Text #4

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.8195. Min Act: -0.1700

Data Index: 3299382 (Open Web Text)

Max Activating Token Index: 431

Text #5

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.8056. Min Act: -0.1700

Data Index: 728863 (Open Web Text)

Max Activating Token Index: 912

Text #6

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.8038. Min Act: -0.1700

Data Index: 7649810 (Open Web Text)

Max Activating Token Index: 376

Text #7

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7852. Min Act: -0.1700

Data Index: 780459 (Open Web Text)

Max Activating Token Index: 616

Text #8

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7765. Min Act: -0.1700

Data Index: 7162239 (Open Web Text)

Max Activating Token Index: 927

Text #9

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7869. Min Act: -0.1700

Data Index: 331708 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7679. Min Act: -0.1700

Data Index: 4031800 (Open Web Text)

Max Activating Token Index: 784

Text #11

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7409. Min Act: -0.1700

Data Index: 8520245 (Open Web Text)

Max Activating Token Index: 783

Text #12

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7722. Min Act: -0.1700

Data Index: 1162412 (Open Web Text)

Max Activating Token Index: 393

Text #13

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7754. Min Act: -0.1700

Data Index: 4216814 (Open Web Text)

Max Activating Token Index: 923

Text #14

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7525. Min Act: -0.1700

Data Index: 5703474 (Open Web Text)

Max Activating Token Index: 893

Text #15

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7467. Min Act: -0.1700

Data Index: 2336277 (Open Web Text)

Max Activating Token Index: 459

Text #16

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7493. Min Act: -0.1700

Data Index: 4339616 (Open Web Text)

Max Activating Token Index: 809

Text #17

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7206. Min Act: -0.1700

Data Index: 6588651 (Open Web Text)

Max Activating Token Index: 253

Text #18

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7076. Min Act: -0.1700

Data Index: 5182024 (Open Web Text)

Max Activating Token Index: 751

Text #19

Max Range: 3.9013. Min Range: -3.9013

Max Act: 3.7368. Min Act: -0.1700

Data Index: 6460902 (Open Web Text)

Max Activating Token Index: 877