Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 326 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.8131. Min Act: -0.1700

Data Index: 2000272 (Open Web Text)

Max Activating Token Index: 471

Text #1

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.7631. Min Act: -0.1700

Data Index: 938782 (Open Web Text)

Max Activating Token Index: 522

Text #2

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.7688. Min Act: -0.1700

Data Index: 719302 (Open Web Text)

Max Activating Token Index: 434

Text #3

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.7470. Min Act: -0.1700

Data Index: 294475 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.7364. Min Act: -0.1700

Data Index: 16552 (Open Web Text)

Max Activating Token Index: 33

Text #5

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.7187. Min Act: -0.1700

Data Index: 5100551 (Open Web Text)

Max Activating Token Index: 36

Text #6

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.7145. Min Act: -0.1700

Data Index: 5659697 (Open Web Text)

Max Activating Token Index: 67

Text #7

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.6865. Min Act: -0.1700

Data Index: 3398885 (Open Web Text)

Max Activating Token Index: 177

Text #8

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.6712. Min Act: -0.1700

Data Index: 2860070 (Open Web Text)

Max Activating Token Index: 52

Text #9

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.6551. Min Act: -0.1700

Data Index: 746478 (Open Web Text)

Max Activating Token Index: 450

Text #10

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.6236. Min Act: -0.1700

Data Index: 4560367 (Open Web Text)

Max Activating Token Index: 994

Text #11

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5896. Min Act: -0.1700

Data Index: 2606067 (Open Web Text)

Max Activating Token Index: 761

Text #12

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5769. Min Act: -0.1700

Data Index: 746479 (Open Web Text)

Max Activating Token Index: 324

Text #13

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5808. Min Act: -0.1700

Data Index: 446018 (Open Web Text)

Max Activating Token Index: 651

Text #14

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5253. Min Act: -0.1700

Data Index: 3529102 (Open Web Text)

Max Activating Token Index: 136

Text #15

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5320. Min Act: -0.1700

Data Index: 8716711 (Open Web Text)

Max Activating Token Index: 2

Text #16

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5685. Min Act: -0.1700

Data Index: 3905411 (Open Web Text)

Max Activating Token Index: 932

Text #17

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5281. Min Act: -0.1700

Data Index: 5444422 (Open Web Text)

Max Activating Token Index: 983

Text #18

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5264. Min Act: -0.1700

Data Index: 3119557 (Open Web Text)

Max Activating Token Index: 490

Text #19

Max Range: 3.8131. Min Range: -3.8131

Max Act: 3.5161. Min Act: -0.1700

Data Index: 4854060 (Open Web Text)

Max Activating Token Index: 478