Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2496 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.8851. Min Act: -0.1700

Data Index: 881772 (Open Web Text)

Max Activating Token Index: 274

Text #1

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.8246. Min Act: -0.1700

Data Index: 4500675 (Open Web Text)

Max Activating Token Index: 593

Text #2

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.8149. Min Act: -0.1700

Data Index: 240835 (Open Web Text)

Max Activating Token Index: 488

Text #3

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.8035. Min Act: -0.1700

Data Index: 6417078 (Open Web Text)

Max Activating Token Index: 485

Text #4

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.8049. Min Act: -0.1700

Data Index: 4950504 (Open Web Text)

Max Activating Token Index: 440

Text #5

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7846. Min Act: -0.1700

Data Index: 5725312 (Open Web Text)

Max Activating Token Index: 561

Text #6

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7800. Min Act: -0.1700

Data Index: 5683582 (Open Web Text)

Max Activating Token Index: 195

Text #7

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7411. Min Act: -0.1700

Data Index: 5754497 (Open Web Text)

Max Activating Token Index: 544

Text #8

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7604. Min Act: -0.1700

Data Index: 1312135 (Open Web Text)

Max Activating Token Index: 510

Text #9

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7499. Min Act: -0.1700

Data Index: 5223012 (Open Web Text)

Max Activating Token Index: 94

Text #10

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7426. Min Act: -0.1700

Data Index: 2900934 (Open Web Text)

Max Activating Token Index: 529

Text #11

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7148. Min Act: -0.1700

Data Index: 1276000 (Open Web Text)

Max Activating Token Index: 491

Text #12

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7266. Min Act: -0.1700

Data Index: 5723551 (Open Web Text)

Max Activating Token Index: 444

Text #13

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.6958. Min Act: -0.1700

Data Index: 5698213 (Open Web Text)

Max Activating Token Index: 419

Text #14

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7130. Min Act: -0.1700

Data Index: 2803334 (Open Web Text)

Max Activating Token Index: 352

Text #15

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.6895. Min Act: -0.1700

Data Index: 2551252 (Open Web Text)

Max Activating Token Index: 460

Text #16

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7084. Min Act: -0.1700

Data Index: 5446673 (Open Web Text)

Max Activating Token Index: 301

Text #17

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7124. Min Act: -0.1700

Data Index: 676683 (Open Web Text)

Max Activating Token Index: 634

Text #18

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.7070. Min Act: -0.1700

Data Index: 664588 (Open Web Text)

Max Activating Token Index: 624

Text #19

Max Range: 5.8851. Min Range: -5.8851

Max Act: 5.6928. Min Act: -0.1700

Data Index: 4689740 (Open Web Text)

Max Activating Token Index: 759