Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2818 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.2521. Min Act: -0.1700

Data Index: 8308101 (Open Web Text)

Max Activating Token Index: 456

Text #1

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.2560. Min Act: -0.1700

Data Index: 7427029 (Open Web Text)

Max Activating Token Index: 921

Text #2

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1899. Min Act: -0.1700

Data Index: 5967981 (Open Web Text)

Max Activating Token Index: 884

Text #3

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1970. Min Act: -0.1700

Data Index: 3688703 (Open Web Text)

Max Activating Token Index: 427

Text #4

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.2006. Min Act: -0.1700

Data Index: 604851 (Open Web Text)

Max Activating Token Index: 450

Text #5

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1487. Min Act: -0.1700

Data Index: 211747 (Open Web Text)

Max Activating Token Index: 919

Text #6

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1350. Min Act: -0.1700

Data Index: 7935040 (Open Web Text)

Max Activating Token Index: 556

Text #7

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1401. Min Act: -0.1700

Data Index: 730980 (Open Web Text)

Max Activating Token Index: 548

Text #8

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1349. Min Act: -0.1700

Data Index: 8183885 (Open Web Text)

Max Activating Token Index: 942

Text #9

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1321. Min Act: -0.1700

Data Index: 5435182 (Open Web Text)

Max Activating Token Index: 267

Text #10

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1370. Min Act: -0.1700

Data Index: 1069102 (Open Web Text)

Max Activating Token Index: 194

Text #11

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1144. Min Act: -0.1700

Data Index: 5145278 (Open Web Text)

Max Activating Token Index: 122

Text #12

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1293. Min Act: -0.1700

Data Index: 8140029 (Open Web Text)

Max Activating Token Index: 92

Text #13

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1249. Min Act: -0.1700

Data Index: 6648072 (Open Web Text)

Max Activating Token Index: 813

Text #14

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1277. Min Act: -0.1700

Data Index: 634266 (Open Web Text)

Max Activating Token Index: 559

Text #15

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1269. Min Act: -0.1700

Data Index: 6631006 (Open Web Text)

Max Activating Token Index: 210

Text #16

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1110. Min Act: -0.1700

Data Index: 3992615 (Open Web Text)

Max Activating Token Index: 413

Text #17

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1053. Min Act: -0.1700

Data Index: 3861206 (Open Web Text)

Max Activating Token Index: 485

Text #18

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.1112. Min Act: -0.1700

Data Index: 1276591 (Open Web Text)

Max Activating Token Index: 867

Text #19

Max Range: 2.2560. Min Range: -2.2560

Max Act: 2.0983. Min Act: -0.1700

Data Index: 2803004 (Open Web Text)

Max Activating Token Index: 378