Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2619 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.8364. Min Act: -0.1700

Data Index: 6925729 (Open Web Text)

Max Activating Token Index: 562

Text #1

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.8564. Min Act: -0.1700

Data Index: 2688351 (Open Web Text)

Max Activating Token Index: 379

Text #2

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7727. Min Act: -0.1700

Data Index: 7799162 (Open Web Text)

Max Activating Token Index: 446

Text #3

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7800. Min Act: -0.1700

Data Index: 2336257 (Open Web Text)

Max Activating Token Index: 697

Text #4

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7487. Min Act: -0.1700

Data Index: 4462504 (Open Web Text)

Max Activating Token Index: 370

Text #5

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7190. Min Act: -0.1700

Data Index: 6994620 (Open Web Text)

Max Activating Token Index: 759

Text #6

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7212. Min Act: -0.1700

Data Index: 6875436 (Open Web Text)

Max Activating Token Index: 207

Text #7

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7450. Min Act: -0.1700

Data Index: 6499816 (Open Web Text)

Max Activating Token Index: 515

Text #8

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7358. Min Act: -0.1700

Data Index: 6147342 (Open Web Text)

Max Activating Token Index: 527

Text #9

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7241. Min Act: -0.1700

Data Index: 6024841 (Open Web Text)

Max Activating Token Index: 320

Text #10

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7264. Min Act: -0.1700

Data Index: 5177532 (Open Web Text)

Max Activating Token Index: 252

Text #11

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7095. Min Act: -0.1700

Data Index: 5013059 (Open Web Text)

Max Activating Token Index: 187

Text #12

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7030. Min Act: -0.1700

Data Index: 5795199 (Open Web Text)

Max Activating Token Index: 238

Text #13

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7091. Min Act: -0.1700

Data Index: 5334358 (Open Web Text)

Max Activating Token Index: 396

Text #14

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.6954. Min Act: -0.1700

Data Index: 1563964 (Open Web Text)

Max Activating Token Index: 196

Text #15

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.6864. Min Act: -0.1700

Data Index: 5019852 (Open Web Text)

Max Activating Token Index: 134

Text #16

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.7000. Min Act: -0.1700

Data Index: 6179844 (Open Web Text)

Max Activating Token Index: 312

Text #17

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.6788. Min Act: -0.1700

Data Index: 11046 (Open Web Text)

Max Activating Token Index: 263

Text #18

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.6938. Min Act: -0.1700

Data Index: 1746527 (Open Web Text)

Max Activating Token Index: 154

Text #19

Max Range: 3.8564. Min Range: -3.8564

Max Act: 3.6833. Min Act: -0.1700

Data Index: 4868528 (Open Web Text)

Max Activating Token Index: 395