Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2265 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.9117. Min Act: -0.1700

Data Index: 4269040 (Open Web Text)

Max Activating Token Index: 58

Text #1

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.9414. Min Act: -0.1700

Data Index: 1934237 (Open Web Text)

Max Activating Token Index: 937

Text #2

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.8616. Min Act: -0.1700

Data Index: 795129 (Open Web Text)

Max Activating Token Index: 126

Text #3

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.8548. Min Act: -0.1700

Data Index: 6215075 (Open Web Text)

Max Activating Token Index: 614

Text #4

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.8139. Min Act: -0.1700

Data Index: 7341174 (Open Web Text)

Max Activating Token Index: 870

Text #5

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.8732. Min Act: -0.1700

Data Index: 973625 (Open Web Text)

Max Activating Token Index: 629

Text #6

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.8530. Min Act: -0.1700

Data Index: 6327270 (Open Web Text)

Max Activating Token Index: 708

Text #7

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.8297. Min Act: -0.1700

Data Index: 1650187 (Open Web Text)

Max Activating Token Index: 536

Text #8

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.8173. Min Act: -0.1700

Data Index: 1007448 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7955. Min Act: -0.1700

Data Index: 4689991 (Open Web Text)

Max Activating Token Index: 769

Text #10

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7612. Min Act: -0.1700

Data Index: 6844493 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7499. Min Act: -0.1700

Data Index: 5515145 (Open Web Text)

Max Activating Token Index: 404

Text #12

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7722. Min Act: -0.1700

Data Index: 5571786 (Open Web Text)

Max Activating Token Index: 812

Text #13

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7822. Min Act: -0.1700

Data Index: 4622516 (Open Web Text)

Max Activating Token Index: 923

Text #14

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7443. Min Act: -0.1700

Data Index: 7765600 (Open Web Text)

Max Activating Token Index: 234

Text #15

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7681. Min Act: -0.1700

Data Index: 4489184 (Open Web Text)

Max Activating Token Index: 440

Text #16

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7527. Min Act: -0.1700

Data Index: 6975432 (Open Web Text)

Max Activating Token Index: 312

Text #17

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7449. Min Act: -0.1700

Data Index: 3454758 (Open Web Text)

Max Activating Token Index: 1005

Text #18

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.7064. Min Act: -0.1700

Data Index: 909903 (Open Web Text)

Max Activating Token Index: 1020

Text #19

Max Range: 3.9414. Min Range: -3.9414

Max Act: 3.6957. Min Act: -0.1700

Data Index: 6760052 (Open Web Text)

Max Activating Token Index: 602