Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 697 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.9113. Min Act: -0.1700

Data Index: 8561719 (Open Web Text)

Max Activating Token Index: 860

Text #1

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.9472. Min Act: -0.1700

Data Index: 2973079 (Open Web Text)

Max Activating Token Index: 652

Text #2

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.8386. Min Act: -0.1700

Data Index: 2919566 (Open Web Text)

Max Activating Token Index: 356

Text #3

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.7570. Min Act: -0.1700

Data Index: 422126 (Open Web Text)

Max Activating Token Index: 410

Text #4

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.7477. Min Act: -0.1700

Data Index: 7006295 (Open Web Text)

Max Activating Token Index: 216

Text #5

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.7320. Min Act: -0.1700

Data Index: 7408004 (Open Web Text)

Max Activating Token Index: 422

Text #6

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.7474. Min Act: -0.1700

Data Index: 2856314 (Open Web Text)

Max Activating Token Index: 878

Text #7

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.7177. Min Act: -0.1700

Data Index: 5868152 (Open Web Text)

Max Activating Token Index: 925

Text #8

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6946. Min Act: -0.1700

Data Index: 7953745 (Open Web Text)

Max Activating Token Index: 458

Text #9

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6996. Min Act: -0.1700

Data Index: 6374428 (Open Web Text)

Max Activating Token Index: 476

Text #10

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6975. Min Act: -0.1700

Data Index: 2919556 (Open Web Text)

Max Activating Token Index: 385

Text #11

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6635. Min Act: -0.1700

Data Index: 4223790 (Open Web Text)

Max Activating Token Index: 327

Text #12

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6554. Min Act: -0.1700

Data Index: 7636280 (Open Web Text)

Max Activating Token Index: 764

Text #13

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6667. Min Act: -0.1700

Data Index: 8235126 (Open Web Text)

Max Activating Token Index: 356

Text #14

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6468. Min Act: -0.1700

Data Index: 7792639 (Open Web Text)

Max Activating Token Index: 854

Text #15

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6607. Min Act: -0.1700

Data Index: 399492 (Open Web Text)

Max Activating Token Index: 472

Text #16

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6525. Min Act: -0.1700

Data Index: 5817513 (Open Web Text)

Max Activating Token Index: 554

Text #17

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6385. Min Act: -0.1700

Data Index: 8568922 (Open Web Text)

Max Activating Token Index: 374

Text #18

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6531. Min Act: -0.1700

Data Index: 6866785 (Open Web Text)

Max Activating Token Index: 391

Text #19

Max Range: 5.9472. Min Range: -5.9472

Max Act: 5.6081. Min Act: -0.1700

Data Index: 3417418 (Open Web Text)

Max Activating Token Index: 351