Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1927 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1581. Min Range: -4.1581

Max Act: 4.1581. Min Act: -0.1700

Data Index: 2483163 (Open Web Text)

Max Activating Token Index: 818

Text #1

Max Range: 4.1581. Min Range: -4.1581

Max Act: 4.1275. Min Act: -0.1700

Data Index: 7361637 (Open Web Text)

Max Activating Token Index: 1022

Text #2

Max Range: 4.1581. Min Range: -4.1581

Max Act: 4.0485. Min Act: -0.1700

Data Index: 903330 (Open Web Text)

Max Activating Token Index: 784

Text #3

Max Range: 4.1581. Min Range: -4.1581

Max Act: 4.0375. Min Act: -0.1700

Data Index: 2465214 (Open Web Text)

Max Activating Token Index: 1005

Text #4

Max Range: 4.1581. Min Range: -4.1581

Max Act: 4.0241. Min Act: -0.1700

Data Index: 6024195 (Open Web Text)

Max Activating Token Index: 731

Text #5

Max Range: 4.1581. Min Range: -4.1581

Max Act: 4.0267. Min Act: -0.1700

Data Index: 231873 (Open Web Text)

Max Activating Token Index: 690

Text #6

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.9940. Min Act: -0.1700

Data Index: 8058400 (Open Web Text)

Max Activating Token Index: 644

Text #7

Max Range: 4.1581. Min Range: -4.1581

Max Act: 4.0046. Min Act: -0.1700

Data Index: 6418119 (Open Web Text)

Max Activating Token Index: 959

Text #8

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.9148. Min Act: -0.1700

Data Index: 2028588 (Open Web Text)

Max Activating Token Index: 670

Text #9

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.9370. Min Act: -0.1700

Data Index: 3285901 (Open Web Text)

Max Activating Token Index: 102

Text #10

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8945. Min Act: -0.1700

Data Index: 1266007 (Open Web Text)

Max Activating Token Index: 176

Text #11

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8944. Min Act: -0.1700

Data Index: 2328734 (Open Web Text)

Max Activating Token Index: 779

Text #12

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8819. Min Act: -0.1700

Data Index: 5518799 (Open Web Text)

Max Activating Token Index: 287

Text #13

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8507. Min Act: -0.1700

Data Index: 4813612 (Open Web Text)

Max Activating Token Index: 382

Text #14

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8498. Min Act: -0.1700

Data Index: 2398611 (Open Web Text)

Max Activating Token Index: 486

Text #15

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8599. Min Act: -0.1700

Data Index: 1855489 (Open Web Text)

Max Activating Token Index: 254

Text #16

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8589. Min Act: -0.1700

Data Index: 5456021 (Open Web Text)

Max Activating Token Index: 1014

Text #17

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.8036. Min Act: -0.1700

Data Index: 8649711 (Open Web Text)

Max Activating Token Index: 299

Text #18

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.7884. Min Act: -0.1700

Data Index: 3940081 (Open Web Text)

Max Activating Token Index: 700

Text #19

Max Range: 4.1581. Min Range: -4.1581

Max Act: 3.7858. Min Act: -0.1700

Data Index: 4681557 (Open Web Text)

Max Activating Token Index: 229