Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 859 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.2469. Min Act: -0.1700

Data Index: 124912 (Open Web Text)

Max Activating Token Index: 1010

Text #1

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1502. Min Act: -0.1700

Data Index: 6223655 (Open Web Text)

Max Activating Token Index: 993

Text #2

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1165. Min Act: -0.1700

Data Index: 3823719 (Open Web Text)

Max Activating Token Index: 1001

Text #3

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1471. Min Act: -0.1700

Data Index: 1404853 (Open Web Text)

Max Activating Token Index: 979

Text #4

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1358. Min Act: -0.1700

Data Index: 4166344 (Open Web Text)

Max Activating Token Index: 951

Text #5

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1008. Min Act: -0.1700

Data Index: 94164 (Open Web Text)

Max Activating Token Index: 730

Text #6

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1046. Min Act: -0.1700

Data Index: 5962205 (Open Web Text)

Max Activating Token Index: 76

Text #7

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1315. Min Act: -0.1700

Data Index: 4493168 (Open Web Text)

Max Activating Token Index: 533

Text #8

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0900. Min Act: -0.1700

Data Index: 807827 (Open Web Text)

Max Activating Token Index: 1020

Text #9

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0830. Min Act: -0.1700

Data Index: 6619133 (Open Web Text)

Max Activating Token Index: 129

Text #10

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0481. Min Act: -0.1700

Data Index: 811392 (Open Web Text)

Max Activating Token Index: 420

Text #11

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0726. Min Act: -0.1700

Data Index: 4160387 (Open Web Text)

Max Activating Token Index: 934

Text #12

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0462. Min Act: -0.1700

Data Index: 5500712 (Open Web Text)

Max Activating Token Index: 996

Text #13

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0535. Min Act: -0.1700

Data Index: 1194490 (Open Web Text)

Max Activating Token Index: 735

Text #14

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0625. Min Act: -0.1700

Data Index: 5829493 (Open Web Text)

Max Activating Token Index: 644

Text #15

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0895. Min Act: -0.1700

Data Index: 3499825 (Open Web Text)

Max Activating Token Index: 323

Text #16

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0635. Min Act: -0.1700

Data Index: 427779 (Open Web Text)

Max Activating Token Index: 157

Text #17

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0675. Min Act: -0.1700

Data Index: 6446271 (Open Web Text)

Max Activating Token Index: 429

Text #18

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0787. Min Act: -0.1700

Data Index: 1978903 (Open Web Text)

Max Activating Token Index: 731

Text #19

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.0251. Min Act: -0.1700

Data Index: 6595317 (Open Web Text)

Max Activating Token Index: 1003