Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2019 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6520. Min Act: -0.1700

Data Index: 908885 (Open Web Text)

Max Activating Token Index: 361

Text #1

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6648. Min Act: -0.1700

Data Index: 7615663 (Open Web Text)

Max Activating Token Index: 589

Text #2

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6600. Min Act: -0.1700

Data Index: 2570138 (Open Web Text)

Max Activating Token Index: 22

Text #3

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6649. Min Act: -0.1700

Data Index: 1973938 (Open Web Text)

Max Activating Token Index: 517

Text #4

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5521. Min Act: -0.1700

Data Index: 7456542 (Open Web Text)

Max Activating Token Index: 485

Text #5

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6535. Min Act: -0.1700

Data Index: 447295 (Open Web Text)

Max Activating Token Index: 516

Text #6

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6343. Min Act: -0.1700

Data Index: 4475797 (Open Web Text)

Max Activating Token Index: 583

Text #7

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5703. Min Act: -0.1700

Data Index: 610045 (Open Web Text)

Max Activating Token Index: 27

Text #8

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5752. Min Act: -0.1700

Data Index: 7291862 (Open Web Text)

Max Activating Token Index: 86

Text #9

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6170. Min Act: -0.1700

Data Index: 7215972 (Open Web Text)

Max Activating Token Index: 402

Text #10

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6061. Min Act: -0.1700

Data Index: 5942603 (Open Web Text)

Max Activating Token Index: 585

Text #11

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5567. Min Act: -0.1700

Data Index: 4474694 (Open Web Text)

Max Activating Token Index: 780

Text #12

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.6133. Min Act: -0.1700

Data Index: 4167120 (Open Web Text)

Max Activating Token Index: 462

Text #13

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5980. Min Act: -0.1700

Data Index: 2688824 (Open Web Text)

Max Activating Token Index: 537

Text #14

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5706. Min Act: -0.1700

Data Index: 7873031 (Open Web Text)

Max Activating Token Index: 465

Text #15

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5647. Min Act: -0.1700

Data Index: 6042924 (Open Web Text)

Max Activating Token Index: 514

Text #16

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5792. Min Act: -0.1700

Data Index: 1478312 (Open Web Text)

Max Activating Token Index: 437

Text #17

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5295. Min Act: -0.1700

Data Index: 3251831 (Open Web Text)

Max Activating Token Index: 39

Text #18

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5403. Min Act: -0.1700

Data Index: 3807098 (Open Web Text)

Max Activating Token Index: 222

Text #19

Max Range: 6.6649. Min Range: -6.6649

Max Act: 6.5549. Min Act: -0.1700

Data Index: 639515 (Open Web Text)

Max Activating Token Index: 889