Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2424 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.1820. Min Act: -0.1700

Data Index: 1044627 (Open Web Text)

Max Activating Token Index: 917

Text #1

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.1411. Min Act: -0.1700

Data Index: 8109458 (Open Web Text)

Max Activating Token Index: 192

Text #2

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.1761. Min Act: -0.1700

Data Index: 7227224 (Open Web Text)

Max Activating Token Index: 353

Text #3

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.1133. Min Act: -0.1700

Data Index: 2733813 (Open Web Text)

Max Activating Token Index: 689

Text #4

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.1058. Min Act: -0.1700

Data Index: 4657639 (Open Web Text)

Max Activating Token Index: 515

Text #5

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.0852. Min Act: -0.1700

Data Index: 7950312 (Open Web Text)

Max Activating Token Index: 466

Text #6

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.1084. Min Act: -0.1700

Data Index: 2211914 (Open Web Text)

Max Activating Token Index: 557

Text #7

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.0315. Min Act: -0.1700

Data Index: 2436254 (Open Web Text)

Max Activating Token Index: 86

Text #8

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.0048. Min Act: -0.1700

Data Index: 1751603 (Open Web Text)

Max Activating Token Index: 334

Text #9

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9570. Min Act: -0.1700

Data Index: 7203278 (Open Web Text)

Max Activating Token Index: 762

Text #10

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.0423. Min Act: -0.1700

Data Index: 6462194 (Open Web Text)

Max Activating Token Index: 396

Text #11

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9675. Min Act: -0.1700

Data Index: 8010198 (Open Web Text)

Max Activating Token Index: 603

Text #12

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9700. Min Act: -0.1700

Data Index: 3274824 (Open Web Text)

Max Activating Token Index: 419

Text #13

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.0044. Min Act: -0.1700

Data Index: 2855799 (Open Web Text)

Max Activating Token Index: 569

Text #14

Max Range: 4.1820. Min Range: -4.1820

Max Act: 4.0046. Min Act: -0.1700

Data Index: 8174423 (Open Web Text)

Max Activating Token Index: 297

Text #15

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9869. Min Act: -0.1700

Data Index: 3316315 (Open Web Text)

Max Activating Token Index: 371

Text #16

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9257. Min Act: -0.1700

Data Index: 8691667 (Open Web Text)

Max Activating Token Index: 588

Text #17

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9459. Min Act: -0.1700

Data Index: 7454803 (Open Web Text)

Max Activating Token Index: 277

Text #18

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9555. Min Act: -0.1700

Data Index: 5796374 (Open Web Text)

Max Activating Token Index: 887

Text #19

Max Range: 4.1820. Min Range: -4.1820

Max Act: 3.9827. Min Act: -0.1700

Data Index: 2567318 (Open Web Text)

Max Activating Token Index: 214