Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2909 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.5061. Min Act: -0.1700

Data Index: 6137506 (Open Web Text)

Max Activating Token Index: 291

Text #1

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.5413. Min Act: -0.1700

Data Index: 2993647 (Open Web Text)

Max Activating Token Index: 699

Text #2

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4372. Min Act: -0.1700

Data Index: 6390442 (Open Web Text)

Max Activating Token Index: 413

Text #3

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4470. Min Act: -0.1700

Data Index: 7678525 (Open Web Text)

Max Activating Token Index: 880

Text #4

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4213. Min Act: -0.1700

Data Index: 7343160 (Open Web Text)

Max Activating Token Index: 1017

Text #5

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4325. Min Act: -0.1700

Data Index: 3942650 (Open Web Text)

Max Activating Token Index: 725

Text #6

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4518. Min Act: -0.1700

Data Index: 5665097 (Open Web Text)

Max Activating Token Index: 307

Text #7

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4346. Min Act: -0.1700

Data Index: 191757 (Open Web Text)

Max Activating Token Index: 178

Text #8

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4075. Min Act: -0.1700

Data Index: 2696324 (Open Web Text)

Max Activating Token Index: 133

Text #9

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4387. Min Act: -0.1700

Data Index: 7990318 (Open Web Text)

Max Activating Token Index: 800

Text #10

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4183. Min Act: -0.1700

Data Index: 434742 (Open Web Text)

Max Activating Token Index: 495

Text #11

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.3981. Min Act: -0.1700

Data Index: 539683 (Open Web Text)

Max Activating Token Index: 322

Text #12

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.3926. Min Act: -0.1700

Data Index: 1088587 (Open Web Text)

Max Activating Token Index: 843

Text #13

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4021. Min Act: -0.1700

Data Index: 8243609 (Open Web Text)

Max Activating Token Index: 961

Text #14

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4123. Min Act: -0.1700

Data Index: 1094430 (Open Web Text)

Max Activating Token Index: 918

Text #15

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.3941. Min Act: -0.1700

Data Index: 5123338 (Open Web Text)

Max Activating Token Index: 12

Text #16

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.3855. Min Act: -0.1700

Data Index: 5578990 (Open Web Text)

Max Activating Token Index: 100

Text #17

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.3383. Min Act: -0.1700

Data Index: 3655401 (Open Web Text)

Max Activating Token Index: 386

Text #18

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.3630. Min Act: -0.1700

Data Index: 1121056 (Open Web Text)

Max Activating Token Index: 193

Text #19

Max Range: 4.5413. Min Range: -4.5413

Max Act: 4.4022. Min Act: -0.1700

Data Index: 996461 (Open Web Text)

Max Activating Token Index: 638