Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2382 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.9215. Min Act: -0.1341

Data Index: 4882826 (Open Web Text)

Max Activating Token Index: 79

Text #1

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.9247. Min Act: -0.1575

Data Index: 5009550 (Open Web Text)

Max Activating Token Index: 85

Text #2

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.9236. Min Act: -0.1548

Data Index: 2946576 (Open Web Text)

Max Activating Token Index: 565

Text #3

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.9073. Min Act: -0.1463

Data Index: 3913699 (Open Web Text)

Max Activating Token Index: 83

Text #4

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.9106. Min Act: -0.1412

Data Index: 3075833 (Open Web Text)

Max Activating Token Index: 85

Text #5

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.9108. Min Act: -0.1558

Data Index: 4298477 (Open Web Text)

Max Activating Token Index: 75

Text #6

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8998. Min Act: -0.1589

Data Index: 1418750 (Open Web Text)

Max Activating Token Index: 89

Text #7

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.9068. Min Act: -0.1534

Data Index: 8048926 (Open Web Text)

Max Activating Token Index: 81

Text #8

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8999. Min Act: -0.1513

Data Index: 6555678 (Open Web Text)

Max Activating Token Index: 86

Text #9

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8930. Min Act: -0.1563

Data Index: 7570209 (Open Web Text)

Max Activating Token Index: 72

Text #10

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8971. Min Act: -0.1291

Data Index: 1859876 (Open Web Text)

Max Activating Token Index: 67

Text #11

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8843. Min Act: -0.1310

Data Index: 2636075 (Open Web Text)

Max Activating Token Index: 613

Text #12

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8837. Min Act: -0.1524

Data Index: 7619276 (Open Web Text)

Max Activating Token Index: 65

Text #13

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8774. Min Act: -0.1411

Data Index: 2072971 (Open Web Text)

Max Activating Token Index: 86

Text #14

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8716. Min Act: -0.1346

Data Index: 7379230 (Open Web Text)

Max Activating Token Index: 569

Text #15

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8741. Min Act: -0.1508

Data Index: 183765 (Open Web Text)

Max Activating Token Index: 61

Text #16

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8762. Min Act: -0.1312

Data Index: 701286 (Open Web Text)

Max Activating Token Index: 564

Text #17

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8774. Min Act: -0.1215

Data Index: 4669458 (Open Web Text)

Max Activating Token Index: 589

Text #18

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8717. Min Act: -0.1494

Data Index: 53722 (Open Web Text)

Max Activating Token Index: 585

Text #19

Max Range: 0.9247. Min Range: -0.9247

Max Act: 0.8662. Min Act: -0.1674

Data Index: 3726707 (Open Web Text)

Max Activating Token Index: 531