Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2289 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4208. Min Range: -4.4208

Max Act: 4.4208. Min Act: -0.1700

Data Index: 1361241 (Open Web Text)

Max Activating Token Index: 331

Text #1

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9971. Min Act: -0.1700

Data Index: 8745463 (Open Web Text)

Max Activating Token Index: 837

Text #2

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9753. Min Act: -0.1700

Data Index: 2506369 (Open Web Text)

Max Activating Token Index: 930

Text #3

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9517. Min Act: -0.1700

Data Index: 5950284 (Open Web Text)

Max Activating Token Index: 354

Text #4

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9399. Min Act: -0.1700

Data Index: 110954 (Open Web Text)

Max Activating Token Index: 932

Text #5

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9218. Min Act: -0.1700

Data Index: 4697424 (Open Web Text)

Max Activating Token Index: 638

Text #6

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9462. Min Act: -0.1700

Data Index: 8463812 (Open Web Text)

Max Activating Token Index: 225

Text #7

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9136. Min Act: -0.1700

Data Index: 7026279 (Open Web Text)

Max Activating Token Index: 738

Text #8

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9237. Min Act: -0.1700

Data Index: 2604011 (Open Web Text)

Max Activating Token Index: 493

Text #9

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9013. Min Act: -0.1700

Data Index: 1776662 (Open Web Text)

Max Activating Token Index: 370

Text #10

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9104. Min Act: -0.1700

Data Index: 1944887 (Open Web Text)

Max Activating Token Index: 361

Text #11

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.9311. Min Act: -0.1700

Data Index: 6618551 (Open Web Text)

Max Activating Token Index: 313

Text #12

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8925. Min Act: -0.1700

Data Index: 5380247 (Open Web Text)

Max Activating Token Index: 52

Text #13

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8446. Min Act: -0.1700

Data Index: 1719362 (Open Web Text)

Max Activating Token Index: 721

Text #14

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8672. Min Act: -0.1700

Data Index: 8298611 (Open Web Text)

Max Activating Token Index: 327

Text #15

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8360. Min Act: -0.1700

Data Index: 195897 (Open Web Text)

Max Activating Token Index: 343

Text #16

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8659. Min Act: -0.1700

Data Index: 5111518 (Open Web Text)

Max Activating Token Index: 306

Text #17

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8577. Min Act: -0.1700

Data Index: 170955 (Open Web Text)

Max Activating Token Index: 239

Text #18

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8511. Min Act: -0.1700

Data Index: 543980 (Open Web Text)

Max Activating Token Index: 454

Text #19

Max Range: 4.4208. Min Range: -4.4208

Max Act: 3.8629. Min Act: -0.1700

Data Index: 7389664 (Open Web Text)

Max Activating Token Index: 111