Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 346 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.4716. Min Act: -0.1700

Data Index: 5242411 (Open Web Text)

Max Activating Token Index: 342

Text #1

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.4718. Min Act: -0.1700

Data Index: 3871596 (Open Web Text)

Max Activating Token Index: 457

Text #2

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.4268. Min Act: -0.1700

Data Index: 4948827 (Open Web Text)

Max Activating Token Index: 989

Text #3

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.4237. Min Act: -0.1700

Data Index: 3580518 (Open Web Text)

Max Activating Token Index: 638

Text #4

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.4337. Min Act: -0.1700

Data Index: 787028 (Open Web Text)

Max Activating Token Index: 270

Text #5

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.4185. Min Act: -0.1700

Data Index: 3066898 (Open Web Text)

Max Activating Token Index: 149

Text #6

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3874. Min Act: -0.1700

Data Index: 7560318 (Open Web Text)

Max Activating Token Index: 961

Text #7

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3928. Min Act: -0.1700

Data Index: 7516540 (Open Web Text)

Max Activating Token Index: 135

Text #8

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3874. Min Act: -0.1700

Data Index: 1420036 (Open Web Text)

Max Activating Token Index: 522

Text #9

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3674. Min Act: -0.1700

Data Index: 6998848 (Open Web Text)

Max Activating Token Index: 771

Text #10

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3652. Min Act: -0.1700

Data Index: 2366994 (Open Web Text)

Max Activating Token Index: 273

Text #11

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3862. Min Act: -0.1700

Data Index: 4454866 (Open Web Text)

Max Activating Token Index: 522

Text #12

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3552. Min Act: -0.1700

Data Index: 8248085 (Open Web Text)

Max Activating Token Index: 630

Text #13

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3695. Min Act: -0.1700

Data Index: 2433075 (Open Web Text)

Max Activating Token Index: 895

Text #14

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3501. Min Act: -0.1700

Data Index: 1361018 (Open Web Text)

Max Activating Token Index: 885

Text #15

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3439. Min Act: -0.1700

Data Index: 7593485 (Open Web Text)

Max Activating Token Index: 111

Text #16

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3489. Min Act: -0.1700

Data Index: 2748829 (Open Web Text)

Max Activating Token Index: 324

Text #17

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3258. Min Act: -0.1700

Data Index: 2030151 (Open Web Text)

Max Activating Token Index: 376

Text #18

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3357. Min Act: -0.1700

Data Index: 7143294 (Open Web Text)

Max Activating Token Index: 25

Text #19

Max Range: 3.4718. Min Range: -3.4718

Max Act: 3.3284. Min Act: -0.1700

Data Index: 1658980 (Open Web Text)

Max Activating Token Index: 435