Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 446 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.5002. Min Act: -0.1700

Data Index: 4031737 (Open Web Text)

Max Activating Token Index: 143

Text #1

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.4521. Min Act: -0.1700

Data Index: 6339648 (Open Web Text)

Max Activating Token Index: 663

Text #2

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3823. Min Act: -0.1700

Data Index: 2370225 (Open Web Text)

Max Activating Token Index: 325

Text #3

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3749. Min Act: -0.1700

Data Index: 3669762 (Open Web Text)

Max Activating Token Index: 575

Text #4

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3929. Min Act: -0.1700

Data Index: 4815132 (Open Web Text)

Max Activating Token Index: 40

Text #5

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3307. Min Act: -0.1700

Data Index: 186006 (Open Web Text)

Max Activating Token Index: 258

Text #6

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3240. Min Act: -0.1700

Data Index: 8338466 (Open Web Text)

Max Activating Token Index: 268

Text #7

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3381. Min Act: -0.1700

Data Index: 388722 (Open Web Text)

Max Activating Token Index: 339

Text #8

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3261. Min Act: -0.1700

Data Index: 3865202 (Open Web Text)

Max Activating Token Index: 221

Text #9

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3216. Min Act: -0.1700

Data Index: 1155263 (Open Web Text)

Max Activating Token Index: 155

Text #10

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3182. Min Act: -0.1700

Data Index: 3473495 (Open Web Text)

Max Activating Token Index: 479

Text #11

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3260. Min Act: -0.1700

Data Index: 4282442 (Open Web Text)

Max Activating Token Index: 69

Text #12

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3355. Min Act: -0.1700

Data Index: 2720127 (Open Web Text)

Max Activating Token Index: 353

Text #13

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.2883. Min Act: -0.1700

Data Index: 7653908 (Open Web Text)

Max Activating Token Index: 394

Text #14

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.2918. Min Act: -0.1700

Data Index: 8223631 (Open Web Text)

Max Activating Token Index: 322

Text #15

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.2993. Min Act: -0.1700

Data Index: 3565635 (Open Web Text)

Max Activating Token Index: 29

Text #16

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.2900. Min Act: -0.1700

Data Index: 4450176 (Open Web Text)

Max Activating Token Index: 481

Text #17

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.3028. Min Act: -0.1700

Data Index: 3377283 (Open Web Text)

Max Activating Token Index: 310

Text #18

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.2995. Min Act: -0.1700

Data Index: 3889158 (Open Web Text)

Max Activating Token Index: 34

Text #19

Max Range: 5.5002. Min Range: -5.5002

Max Act: 5.2822. Min Act: -0.1700

Data Index: 8464731 (Open Web Text)

Max Activating Token Index: 267