Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2225 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.6677. Min Act: -0.1700

Data Index: 1903748 (Open Web Text)

Max Activating Token Index: 423

Text #1

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.5665. Min Act: -0.1700

Data Index: 2512466 (Open Web Text)

Max Activating Token Index: 336

Text #2

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.5075. Min Act: -0.1700

Data Index: 3175759 (Open Web Text)

Max Activating Token Index: 501

Text #3

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4794. Min Act: -0.1700

Data Index: 2136648 (Open Web Text)

Max Activating Token Index: 610

Text #4

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4555. Min Act: -0.1700

Data Index: 1229538 (Open Web Text)

Max Activating Token Index: 817

Text #5

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4362. Min Act: -0.1700

Data Index: 1691738 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4551. Min Act: -0.1700

Data Index: 1849693 (Open Web Text)

Max Activating Token Index: 615

Text #7

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4152. Min Act: -0.1700

Data Index: 3104195 (Open Web Text)

Max Activating Token Index: 583

Text #8

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4240. Min Act: -0.1700

Data Index: 5882713 (Open Web Text)

Max Activating Token Index: 535

Text #9

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4251. Min Act: -0.1700

Data Index: 6070767 (Open Web Text)

Max Activating Token Index: 214

Text #10

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4115. Min Act: -0.1700

Data Index: 7207934 (Open Web Text)

Max Activating Token Index: 703

Text #11

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3941. Min Act: -0.1700

Data Index: 1294644 (Open Web Text)

Max Activating Token Index: 479

Text #12

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3792. Min Act: -0.1700

Data Index: 2567567 (Open Web Text)

Max Activating Token Index: 941

Text #13

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3786. Min Act: -0.1700

Data Index: 692169 (Open Web Text)

Max Activating Token Index: 682

Text #14

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4133. Min Act: -0.1700

Data Index: 5863969 (Open Web Text)

Max Activating Token Index: 513

Text #15

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3864. Min Act: -0.1700

Data Index: 4120040 (Open Web Text)

Max Activating Token Index: 240

Text #16

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4131. Min Act: -0.1700

Data Index: 3232329 (Open Web Text)

Max Activating Token Index: 722

Text #17

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3591. Min Act: -0.1700

Data Index: 2224864 (Open Web Text)

Max Activating Token Index: 415

Text #18

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3553. Min Act: -0.1700

Data Index: 2128911 (Open Web Text)

Max Activating Token Index: 590

Text #19

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3489. Min Act: -0.1700

Data Index: 2885452 (Open Web Text)

Max Activating Token Index: 151