Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1312 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8856. Min Act: -0.1700

Data Index: 1656864 (Open Web Text)

Max Activating Token Index: 226

Text #1

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8608. Min Act: -0.1700

Data Index: 2366590 (Open Web Text)

Max Activating Token Index: 310

Text #2

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8629. Min Act: -0.1700

Data Index: 5155154 (Open Web Text)

Max Activating Token Index: 336

Text #3

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8366. Min Act: -0.1700

Data Index: 1899360 (Open Web Text)

Max Activating Token Index: 1000

Text #4

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8300. Min Act: -0.1700

Data Index: 258784 (Open Web Text)

Max Activating Token Index: 1012

Text #5

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8412. Min Act: -0.1700

Data Index: 1861812 (Open Web Text)

Max Activating Token Index: 662

Text #6

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8442. Min Act: -0.1700

Data Index: 6634358 (Open Web Text)

Max Activating Token Index: 1008

Text #7

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8198. Min Act: -0.1700

Data Index: 7443197 (Open Web Text)

Max Activating Token Index: 610

Text #8

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8141. Min Act: -0.1700

Data Index: 6045823 (Open Web Text)

Max Activating Token Index: 829

Text #9

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8346. Min Act: -0.1700

Data Index: 4436761 (Open Web Text)

Max Activating Token Index: 1018

Text #10

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8299. Min Act: -0.1700

Data Index: 5366492 (Open Web Text)

Max Activating Token Index: 676

Text #11

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8104. Min Act: -0.1700

Data Index: 1152035 (Open Web Text)

Max Activating Token Index: 716

Text #12

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8063. Min Act: -0.1700

Data Index: 7802616 (Open Web Text)

Max Activating Token Index: 1022

Text #13

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8262. Min Act: -0.1700

Data Index: 304923 (Open Web Text)

Max Activating Token Index: 469

Text #14

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.7913. Min Act: -0.1700

Data Index: 4304094 (Open Web Text)

Max Activating Token Index: 1007

Text #15

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8091. Min Act: -0.1700

Data Index: 4864212 (Open Web Text)

Max Activating Token Index: 954

Text #16

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8134. Min Act: -0.1700

Data Index: 7723156 (Open Web Text)

Max Activating Token Index: 196

Text #17

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8056. Min Act: -0.1700

Data Index: 7985351 (Open Web Text)

Max Activating Token Index: 168

Text #18

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8103. Min Act: -0.1700

Data Index: 2237478 (Open Web Text)

Max Activating Token Index: 722

Text #19

Max Range: 1.8856. Min Range: -1.8856

Max Act: 1.8105. Min Act: -0.1700

Data Index: 2472378 (Open Web Text)

Max Activating Token Index: 111