Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2414 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0758. Min Range: -3.0758

Max Act: 3.0758. Min Act: -0.1700

Data Index: 4100096 (Open Web Text)

Max Activating Token Index: 964

Text #1

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.9767. Min Act: -0.1700

Data Index: 5381361 (Open Web Text)

Max Activating Token Index: 493

Text #2

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.9182. Min Act: -0.1700

Data Index: 2576629 (Open Web Text)

Max Activating Token Index: 975

Text #3

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8461. Min Act: -0.1700

Data Index: 8573722 (Open Web Text)

Max Activating Token Index: 706

Text #4

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8316. Min Act: -0.1700

Data Index: 6618990 (Open Web Text)

Max Activating Token Index: 831

Text #5

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8407. Min Act: -0.1700

Data Index: 3224009 (Open Web Text)

Max Activating Token Index: 533

Text #6

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8591. Min Act: -0.1700

Data Index: 3191410 (Open Web Text)

Max Activating Token Index: 463

Text #7

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8441. Min Act: -0.1700

Data Index: 4157268 (Open Web Text)

Max Activating Token Index: 653

Text #8

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8359. Min Act: -0.1700

Data Index: 1935595 (Open Web Text)

Max Activating Token Index: 989

Text #9

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8285. Min Act: -0.1700

Data Index: 3924273 (Open Web Text)

Max Activating Token Index: 113

Text #10

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8405. Min Act: -0.1700

Data Index: 2488084 (Open Web Text)

Max Activating Token Index: 418

Text #11

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8130. Min Act: -0.1700

Data Index: 7644054 (Open Web Text)

Max Activating Token Index: 348

Text #12

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8194. Min Act: -0.1700

Data Index: 2915341 (Open Web Text)

Max Activating Token Index: 354

Text #13

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8128. Min Act: -0.1700

Data Index: 2252607 (Open Web Text)

Max Activating Token Index: 627

Text #14

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8091. Min Act: -0.1700

Data Index: 8131750 (Open Web Text)

Max Activating Token Index: 140

Text #15

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.7954. Min Act: -0.1700

Data Index: 4201276 (Open Web Text)

Max Activating Token Index: 890

Text #16

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.7971. Min Act: -0.1700

Data Index: 8029917 (Open Web Text)

Max Activating Token Index: 193

Text #17

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.7903. Min Act: -0.1700

Data Index: 2045027 (Open Web Text)

Max Activating Token Index: 410

Text #18

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.8047. Min Act: -0.1700

Data Index: 5786856 (Open Web Text)

Max Activating Token Index: 518

Text #19

Max Range: 3.0758. Min Range: -3.0758

Max Act: 2.7779. Min Act: -0.1700

Data Index: 282820 (Open Web Text)

Max Activating Token Index: 703