Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2482 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0755. Min Range: -6.0755

Max Act: 6.0755. Min Act: -0.1700

Data Index: 2733931 (Open Web Text)

Max Activating Token Index: 317

Text #1

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.7552. Min Act: -0.1700

Data Index: 8405858 (Open Web Text)

Max Activating Token Index: 344

Text #2

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.7474. Min Act: -0.1700

Data Index: 4661572 (Open Web Text)

Max Activating Token Index: 484

Text #3

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.7478. Min Act: -0.1700

Data Index: 7075364 (Open Web Text)

Max Activating Token Index: 832

Text #4

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.7641. Min Act: -0.1700

Data Index: 186849 (Open Web Text)

Max Activating Token Index: 1012

Text #5

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.7122. Min Act: -0.1700

Data Index: 2106616 (Open Web Text)

Max Activating Token Index: 449

Text #6

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6776. Min Act: -0.1700

Data Index: 6801802 (Open Web Text)

Max Activating Token Index: 895

Text #7

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6510. Min Act: -0.1700

Data Index: 540416 (Open Web Text)

Max Activating Token Index: 27

Text #8

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6842. Min Act: -0.1700

Data Index: 596070 (Open Web Text)

Max Activating Token Index: 787

Text #9

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6894. Min Act: -0.1700

Data Index: 5672322 (Open Web Text)

Max Activating Token Index: 345

Text #10

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.5555. Min Act: -0.1700

Data Index: 2246276 (Open Web Text)

Max Activating Token Index: 676

Text #11

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6283. Min Act: -0.1700

Data Index: 7897898 (Open Web Text)

Max Activating Token Index: 121

Text #12

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6233. Min Act: -0.1700

Data Index: 3383113 (Open Web Text)

Max Activating Token Index: 256

Text #13

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6236. Min Act: -0.1700

Data Index: 3838325 (Open Web Text)

Max Activating Token Index: 376

Text #14

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6178. Min Act: -0.1700

Data Index: 1357602 (Open Web Text)

Max Activating Token Index: 999

Text #15

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6160. Min Act: -0.1700

Data Index: 1641462 (Open Web Text)

Max Activating Token Index: 325

Text #16

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.5894. Min Act: -0.1700

Data Index: 706926 (Open Web Text)

Max Activating Token Index: 157

Text #17

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.5958. Min Act: -0.1700

Data Index: 2679903 (Open Web Text)

Max Activating Token Index: 652

Text #18

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.5771. Min Act: -0.1700

Data Index: 7604970 (Open Web Text)

Max Activating Token Index: 335

Text #19

Max Range: 6.0755. Min Range: -6.0755

Max Act: 5.6103. Min Act: -0.1700

Data Index: 5941842 (Open Web Text)

Max Activating Token Index: 446