Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2458 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.9485. Min Act: -0.1700

Data Index: 4834236 (Open Web Text)

Max Activating Token Index: 675

Text #1

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.9629. Min Act: -0.1700

Data Index: 5069537 (Open Web Text)

Max Activating Token Index: 61

Text #2

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.8306. Min Act: -0.1700

Data Index: 3907867 (Open Web Text)

Max Activating Token Index: 757

Text #3

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.8144. Min Act: -0.1700

Data Index: 5726724 (Open Web Text)

Max Activating Token Index: 742

Text #4

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7932. Min Act: -0.1700

Data Index: 6136098 (Open Web Text)

Max Activating Token Index: 267

Text #5

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7548. Min Act: -0.1700

Data Index: 5710279 (Open Web Text)

Max Activating Token Index: 559

Text #6

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7545. Min Act: -0.1700

Data Index: 2695390 (Open Web Text)

Max Activating Token Index: 1019

Text #7

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7200. Min Act: -0.1700

Data Index: 1725964 (Open Web Text)

Max Activating Token Index: 329

Text #8

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7631. Min Act: -0.1700

Data Index: 4778990 (Open Web Text)

Max Activating Token Index: 645

Text #9

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7081. Min Act: -0.1700

Data Index: 4501479 (Open Web Text)

Max Activating Token Index: 94

Text #10

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7043. Min Act: -0.1700

Data Index: 1486111 (Open Web Text)

Max Activating Token Index: 934

Text #11

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.7116. Min Act: -0.1700

Data Index: 1467623 (Open Web Text)

Max Activating Token Index: 771

Text #12

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6860. Min Act: -0.1700

Data Index: 6553131 (Open Web Text)

Max Activating Token Index: 675

Text #13

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6844. Min Act: -0.1700

Data Index: 6440310 (Open Web Text)

Max Activating Token Index: 801

Text #14

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6780. Min Act: -0.1700

Data Index: 5804029 (Open Web Text)

Max Activating Token Index: 945

Text #15

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6190. Min Act: -0.1700

Data Index: 6341781 (Open Web Text)

Max Activating Token Index: 72

Text #16

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6722. Min Act: -0.1700

Data Index: 1934802 (Open Web Text)

Max Activating Token Index: 428

Text #17

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6139. Min Act: -0.1700

Data Index: 978227 (Open Web Text)

Max Activating Token Index: 395

Text #18

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6378. Min Act: -0.1700

Data Index: 434945 (Open Web Text)

Max Activating Token Index: 957

Text #19

Max Range: 4.9629. Min Range: -4.9629

Max Act: 4.6183. Min Act: -0.1700

Data Index: 1036504 (Open Web Text)

Max Activating Token Index: 278