Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1436 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.9070. Min Act: -0.1410

Data Index: 6667031 (Open Web Text)

Max Activating Token Index: 366

Text #1

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8517. Min Act: -0.0421

Data Index: 6667030 (Open Web Text)

Max Activating Token Index: 992

Text #2

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8491. Min Act: -0.0421

Data Index: 7450670 (Open Web Text)

Max Activating Token Index: 407

Text #3

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8474. Min Act: -0.0421

Data Index: 1025872 (Open Web Text)

Max Activating Token Index: 384

Text #4

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8558. Min Act: -0.0421

Data Index: 8789915 (Open Web Text)

Max Activating Token Index: 949

Text #5

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8461. Min Act: -0.0421

Data Index: 5449080 (Open Web Text)

Max Activating Token Index: 400

Text #6

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8420. Min Act: -0.0786

Data Index: 744254 (Open Web Text)

Max Activating Token Index: 959

Text #7

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8395. Min Act: -0.0421

Data Index: 2265719 (Open Web Text)

Max Activating Token Index: 949

Text #8

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8423. Min Act: -0.0421

Data Index: 6813201 (Open Web Text)

Max Activating Token Index: 957

Text #9

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8316. Min Act: -0.0421

Data Index: 4569719 (Open Web Text)

Max Activating Token Index: 1018

Text #10

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8388. Min Act: -0.0421

Data Index: 5868568 (Open Web Text)

Max Activating Token Index: 945

Text #11

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8405. Min Act: -0.0421

Data Index: 6942901 (Open Web Text)

Max Activating Token Index: 396

Text #12

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8371. Min Act: -0.0598

Data Index: 2950629 (Open Web Text)

Max Activating Token Index: 387

Text #13

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8372. Min Act: -0.0421

Data Index: 3968780 (Open Web Text)

Max Activating Token Index: 959

Text #14

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8320. Min Act: -0.0421

Data Index: 2484119 (Open Web Text)

Max Activating Token Index: 375

Text #15

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8394. Min Act: -0.0421

Data Index: 1095886 (Open Web Text)

Max Activating Token Index: 944

Text #16

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8306. Min Act: -0.0507

Data Index: 5788534 (Open Web Text)

Max Activating Token Index: 403

Text #17

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8254. Min Act: -0.0945

Data Index: 5868567 (Open Web Text)

Max Activating Token Index: 971

Text #18

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8310. Min Act: -0.0448

Data Index: 5198793 (Open Web Text)

Max Activating Token Index: 1012

Text #19

Max Range: 0.9070. Min Range: -0.9070

Max Act: 0.8346. Min Act: -0.0421

Data Index: 5235270 (Open Web Text)

Max Activating Token Index: 990