Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3004 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2670. Min Range: -3.2670

Max Act: 3.2670. Min Act: -0.1700

Data Index: 2514450 (Open Web Text)

Max Activating Token Index: 530

Text #1

Max Range: 3.2670. Min Range: -3.2670

Max Act: 3.2368. Min Act: -0.1700

Data Index: 4210382 (Open Web Text)

Max Activating Token Index: 1011

Text #2

Max Range: 3.2670. Min Range: -3.2670

Max Act: 3.0409. Min Act: -0.1700

Data Index: 1358975 (Open Web Text)

Max Activating Token Index: 372

Text #3

Max Range: 3.2670. Min Range: -3.2670

Max Act: 3.0094. Min Act: -0.1700

Data Index: 8342977 (Open Web Text)

Max Activating Token Index: 900

Text #4

Max Range: 3.2670. Min Range: -3.2670

Max Act: 3.0378. Min Act: -0.1700

Data Index: 4463282 (Open Web Text)

Max Activating Token Index: 720

Text #5

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9415. Min Act: -0.1700

Data Index: 7914960 (Open Web Text)

Max Activating Token Index: 383

Text #6

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9380. Min Act: -0.1700

Data Index: 5423803 (Open Web Text)

Max Activating Token Index: 528

Text #7

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9440. Min Act: -0.1700

Data Index: 1803216 (Open Web Text)

Max Activating Token Index: 319

Text #8

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9180. Min Act: -0.1700

Data Index: 1464628 (Open Web Text)

Max Activating Token Index: 931

Text #9

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.8871. Min Act: -0.1700

Data Index: 5438684 (Open Web Text)

Max Activating Token Index: 939

Text #10

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.8749. Min Act: -0.1700

Data Index: 8414601 (Open Web Text)

Max Activating Token Index: 425

Text #11

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9276. Min Act: -0.1700

Data Index: 1019755 (Open Web Text)

Max Activating Token Index: 323

Text #12

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9465. Min Act: -0.1700

Data Index: 631015 (Open Web Text)

Max Activating Token Index: 543

Text #13

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9266. Min Act: -0.1700

Data Index: 1727626 (Open Web Text)

Max Activating Token Index: 711

Text #14

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.9098. Min Act: -0.1700

Data Index: 6007783 (Open Web Text)

Max Activating Token Index: 914

Text #15

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.8866. Min Act: -0.1700

Data Index: 1019756 (Open Web Text)

Max Activating Token Index: 308

Text #16

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.8469. Min Act: -0.1700

Data Index: 3823626 (Open Web Text)

Max Activating Token Index: 189

Text #17

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.8900. Min Act: -0.1700

Data Index: 7431389 (Open Web Text)

Max Activating Token Index: 557

Text #18

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.8715. Min Act: -0.1700

Data Index: 5380470 (Open Web Text)

Max Activating Token Index: 929

Text #19

Max Range: 3.2670. Min Range: -3.2670

Max Act: 2.8623. Min Act: -0.1700

Data Index: 2877298 (Open Web Text)

Max Activating Token Index: 560