Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3030 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.4870. Min Act: -0.1700

Data Index: 6405766 (Open Web Text)

Max Activating Token Index: 877

Text #1

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.3199. Min Act: -0.1700

Data Index: 8448113 (Open Web Text)

Max Activating Token Index: 85

Text #2

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.2917. Min Act: -0.1700

Data Index: 3266566 (Open Web Text)

Max Activating Token Index: 190

Text #3

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.2436. Min Act: -0.1700

Data Index: 4935229 (Open Web Text)

Max Activating Token Index: 46

Text #4

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.2196. Min Act: -0.1700

Data Index: 5400485 (Open Web Text)

Max Activating Token Index: 191

Text #5

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1709. Min Act: -0.1700

Data Index: 32108 (Open Web Text)

Max Activating Token Index: 756

Text #6

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1790. Min Act: -0.1700

Data Index: 6298792 (Open Web Text)

Max Activating Token Index: 343

Text #7

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1980. Min Act: -0.1700

Data Index: 1703375 (Open Web Text)

Max Activating Token Index: 435

Text #8

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1345. Min Act: -0.1700

Data Index: 8809956 (Open Web Text)

Max Activating Token Index: 611

Text #9

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1322. Min Act: -0.1700

Data Index: 1539762 (Open Web Text)

Max Activating Token Index: 709

Text #10

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1190. Min Act: -0.1699

Data Index: 6461165 (Open Web Text)

Max Activating Token Index: 819

Text #11

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1480. Min Act: -0.1700

Data Index: 3912781 (Open Web Text)

Max Activating Token Index: 612

Text #12

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1103. Min Act: -0.1700

Data Index: 1017963 (Open Web Text)

Max Activating Token Index: 939

Text #13

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.1052. Min Act: -0.1700

Data Index: 3859616 (Open Web Text)

Max Activating Token Index: 917

Text #14

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.0873. Min Act: -0.1700

Data Index: 5160696 (Open Web Text)

Max Activating Token Index: 729

Text #15

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.0891. Min Act: -0.1700

Data Index: 7049732 (Open Web Text)

Max Activating Token Index: 305

Text #16

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.0571. Min Act: -0.1700

Data Index: 2952828 (Open Web Text)

Max Activating Token Index: 373

Text #17

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.0716. Min Act: -0.1700

Data Index: 7301577 (Open Web Text)

Max Activating Token Index: 858

Text #18

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.0794. Min Act: -0.1700

Data Index: 3359094 (Open Web Text)

Max Activating Token Index: 465

Text #19

Max Range: 5.4870. Min Range: -5.4870

Max Act: 5.0720. Min Act: -0.1700

Data Index: 7866318 (Open Web Text)

Max Activating Token Index: 702