Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1519 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0773. Min Range: -5.0773

Max Act: 5.0773. Min Act: -0.1700

Data Index: 621890 (Open Web Text)

Max Activating Token Index: 820

Text #1

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.9887. Min Act: -0.1700

Data Index: 5430210 (Open Web Text)

Max Activating Token Index: 799

Text #2

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.8629. Min Act: -0.1700

Data Index: 6545157 (Open Web Text)

Max Activating Token Index: 298

Text #3

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.8396. Min Act: -0.1700

Data Index: 467563 (Open Web Text)

Max Activating Token Index: 477

Text #4

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.8236. Min Act: -0.1700

Data Index: 8037959 (Open Web Text)

Max Activating Token Index: 414

Text #5

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.7853. Min Act: -0.1700

Data Index: 4719907 (Open Web Text)

Max Activating Token Index: 107

Text #6

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.7686. Min Act: -0.1700

Data Index: 4395580 (Open Web Text)

Max Activating Token Index: 27

Text #7

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.7241. Min Act: -0.1700

Data Index: 4107549 (Open Web Text)

Max Activating Token Index: 801

Text #8

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6686. Min Act: -0.1700

Data Index: 3516676 (Open Web Text)

Max Activating Token Index: 516

Text #9

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6074. Min Act: -0.1700

Data Index: 4749054 (Open Web Text)

Max Activating Token Index: 571

Text #10

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6325. Min Act: -0.1700

Data Index: 4281643 (Open Web Text)

Max Activating Token Index: 775

Text #11

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6059. Min Act: -0.1700

Data Index: 818540 (Open Web Text)

Max Activating Token Index: 462

Text #12

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6065. Min Act: -0.1700

Data Index: 3018283 (Open Web Text)

Max Activating Token Index: 239

Text #13

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6013. Min Act: -0.1700

Data Index: 6083683 (Open Web Text)

Max Activating Token Index: 50

Text #14

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6049. Min Act: -0.1700

Data Index: 2582914 (Open Web Text)

Max Activating Token Index: 176

Text #15

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.6095. Min Act: -0.1700

Data Index: 1075392 (Open Web Text)

Max Activating Token Index: 624

Text #16

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.5649. Min Act: -0.1700

Data Index: 8107245 (Open Web Text)

Max Activating Token Index: 624

Text #17

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.5506. Min Act: -0.1700

Data Index: 7843292 (Open Web Text)

Max Activating Token Index: 10

Text #18

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.5394. Min Act: -0.1700

Data Index: 6172199 (Open Web Text)

Max Activating Token Index: 717

Text #19

Max Range: 5.0773. Min Range: -5.0773

Max Act: 4.5597. Min Act: -0.1700

Data Index: 6400955 (Open Web Text)

Max Activating Token Index: 14