Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1550 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.6333. Min Act: -0.1700

Data Index: 4418468 (Open Web Text)

Max Activating Token Index: 530

Text #1

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.5321. Min Act: -0.1700

Data Index: 8156664 (Open Web Text)

Max Activating Token Index: 527

Text #2

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.5189. Min Act: -0.1700

Data Index: 1751776 (Open Web Text)

Max Activating Token Index: 855

Text #3

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.4854. Min Act: -0.1700

Data Index: 5822406 (Open Web Text)

Max Activating Token Index: 338

Text #4

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.4908. Min Act: -0.1700

Data Index: 2098977 (Open Web Text)

Max Activating Token Index: 930

Text #5

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.4205. Min Act: -0.1700

Data Index: 1442832 (Open Web Text)

Max Activating Token Index: 156

Text #6

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.4130. Min Act: -0.1700

Data Index: 1973587 (Open Web Text)

Max Activating Token Index: 426

Text #7

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.4078. Min Act: -0.1700

Data Index: 264849 (Open Web Text)

Max Activating Token Index: 216

Text #8

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3988. Min Act: -0.1700

Data Index: 7842756 (Open Web Text)

Max Activating Token Index: 516

Text #9

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3654. Min Act: -0.1700

Data Index: 7957913 (Open Web Text)

Max Activating Token Index: 199

Text #10

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3643. Min Act: -0.1700

Data Index: 6346167 (Open Web Text)

Max Activating Token Index: 362

Text #11

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3281. Min Act: -0.1700

Data Index: 3728058 (Open Web Text)

Max Activating Token Index: 226

Text #12

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.2974. Min Act: -0.1700

Data Index: 2118460 (Open Web Text)

Max Activating Token Index: 739

Text #13

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3421. Min Act: -0.1700

Data Index: 1132399 (Open Web Text)

Max Activating Token Index: 97

Text #14

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3321. Min Act: -0.1700

Data Index: 2186909 (Open Web Text)

Max Activating Token Index: 484

Text #15

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3343. Min Act: -0.1700

Data Index: 8540509 (Open Web Text)

Max Activating Token Index: 957

Text #16

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3213. Min Act: -0.1700

Data Index: 613431 (Open Web Text)

Max Activating Token Index: 307

Text #17

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3428. Min Act: -0.1700

Data Index: 5175808 (Open Web Text)

Max Activating Token Index: 670

Text #18

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.3118. Min Act: -0.1700

Data Index: 2535748 (Open Web Text)

Max Activating Token Index: 168

Text #19

Max Range: 3.6333. Min Range: -3.6333

Max Act: 3.2964. Min Act: -0.1700

Data Index: 7881810 (Open Web Text)

Max Activating Token Index: 732