Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2001 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.2597. Min Act: -0.1700

Data Index: 4558550 (Open Web Text)

Max Activating Token Index: 670

Text #1

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.2493. Min Act: -0.1700

Data Index: 5834771 (Open Web Text)

Max Activating Token Index: 827

Text #2

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.2198. Min Act: -0.1700

Data Index: 2163641 (Open Web Text)

Max Activating Token Index: 120

Text #3

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0734. Min Act: -0.1700

Data Index: 5128490 (Open Web Text)

Max Activating Token Index: 43

Text #4

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0605. Min Act: -0.1700

Data Index: 1850374 (Open Web Text)

Max Activating Token Index: 563

Text #5

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0604. Min Act: -0.1700

Data Index: 6622760 (Open Web Text)

Max Activating Token Index: 26

Text #6

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0890. Min Act: -0.1700

Data Index: 4559643 (Open Web Text)

Max Activating Token Index: 403

Text #7

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0096. Min Act: -0.1700

Data Index: 7772043 (Open Web Text)

Max Activating Token Index: 965

Text #8

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0292. Min Act: -0.1700

Data Index: 901614 (Open Web Text)

Max Activating Token Index: 826

Text #9

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0008. Min Act: -0.1700

Data Index: 138448 (Open Web Text)

Max Activating Token Index: 884

Text #10

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0338. Min Act: -0.1700

Data Index: 5262000 (Open Web Text)

Max Activating Token Index: 17

Text #11

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0291. Min Act: -0.1700

Data Index: 6925597 (Open Web Text)

Max Activating Token Index: 359

Text #12

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0127. Min Act: -0.1700

Data Index: 3829611 (Open Web Text)

Max Activating Token Index: 119

Text #13

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0023. Min Act: -0.1700

Data Index: 1030912 (Open Web Text)

Max Activating Token Index: 447

Text #14

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0093. Min Act: -0.1700

Data Index: 6209300 (Open Web Text)

Max Activating Token Index: 277

Text #15

Max Range: 3.2597. Min Range: -3.2597

Max Act: 3.0106. Min Act: -0.1700

Data Index: 7410135 (Open Web Text)

Max Activating Token Index: 391

Text #16

Max Range: 3.2597. Min Range: -3.2597

Max Act: 2.9567. Min Act: -0.1700

Data Index: 3624584 (Open Web Text)

Max Activating Token Index: 373

Text #17

Max Range: 3.2597. Min Range: -3.2597

Max Act: 2.9596. Min Act: -0.1700

Data Index: 4995853 (Open Web Text)

Max Activating Token Index: 591

Text #18

Max Range: 3.2597. Min Range: -3.2597

Max Act: 2.9844. Min Act: -0.1700

Data Index: 8013777 (Open Web Text)

Max Activating Token Index: 341

Text #19

Max Range: 3.2597. Min Range: -3.2597

Max Act: 2.9799. Min Act: -0.1700

Data Index: 3556519 (Open Web Text)

Max Activating Token Index: 479