Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 194 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.7398. Min Act: -0.1700

Data Index: 7513878 (Open Web Text)

Max Activating Token Index: 417

Text #1

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.7334. Min Act: -0.1700

Data Index: 282454 (Open Web Text)

Max Activating Token Index: 506

Text #2

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.7015. Min Act: -0.1700

Data Index: 6103578 (Open Web Text)

Max Activating Token Index: 68

Text #3

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6888. Min Act: -0.1700

Data Index: 6406144 (Open Web Text)

Max Activating Token Index: 654

Text #4

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6808. Min Act: -0.1700

Data Index: 4773140 (Open Web Text)

Max Activating Token Index: 56

Text #5

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6797. Min Act: -0.1700

Data Index: 3444895 (Open Web Text)

Max Activating Token Index: 134

Text #6

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6744. Min Act: -0.1700

Data Index: 5852546 (Open Web Text)

Max Activating Token Index: 722

Text #7

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6594. Min Act: -0.1700

Data Index: 4309842 (Open Web Text)

Max Activating Token Index: 216

Text #8

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6349. Min Act: -0.1700

Data Index: 4873458 (Open Web Text)

Max Activating Token Index: 693

Text #9

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6475. Min Act: -0.1700

Data Index: 4709447 (Open Web Text)

Max Activating Token Index: 768

Text #10

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6513. Min Act: -0.1700

Data Index: 2699004 (Open Web Text)

Max Activating Token Index: 581

Text #11

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6501. Min Act: -0.1700

Data Index: 374451 (Open Web Text)

Max Activating Token Index: 788

Text #12

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6229. Min Act: -0.1700

Data Index: 4822264 (Open Web Text)

Max Activating Token Index: 719

Text #13

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6074. Min Act: -0.1700

Data Index: 3492441 (Open Web Text)

Max Activating Token Index: 309

Text #14

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6276. Min Act: -0.1700

Data Index: 4092597 (Open Web Text)

Max Activating Token Index: 655

Text #15

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6096. Min Act: -0.1700

Data Index: 8411631 (Open Web Text)

Max Activating Token Index: 201

Text #16

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6444. Min Act: -0.1700

Data Index: 5798203 (Open Web Text)

Max Activating Token Index: 971

Text #17

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6155. Min Act: -0.1700

Data Index: 6573225 (Open Web Text)

Max Activating Token Index: 255

Text #18

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.5946. Min Act: -0.1700

Data Index: 3285300 (Open Web Text)

Max Activating Token Index: 602

Text #19

Max Range: 1.7398. Min Range: -1.7398

Max Act: 1.6077. Min Act: -0.1700

Data Index: 3982496 (Open Web Text)

Max Activating Token Index: 510