Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1433 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.6537. Min Act: -0.1700

Data Index: 7601654 (Open Web Text)

Max Activating Token Index: 32

Text #1

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.6089. Min Act: -0.1700

Data Index: 6916788 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.6022. Min Act: -0.1700

Data Index: 6008944 (Open Web Text)

Max Activating Token Index: 236

Text #3

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5727. Min Act: -0.1700

Data Index: 3894241 (Open Web Text)

Max Activating Token Index: 396

Text #4

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5921. Min Act: -0.1700

Data Index: 41049 (Open Web Text)

Max Activating Token Index: 282

Text #5

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5424. Min Act: -0.1700

Data Index: 7726880 (Open Web Text)

Max Activating Token Index: 33

Text #6

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5694. Min Act: -0.1700

Data Index: 6038193 (Open Web Text)

Max Activating Token Index: 25

Text #7

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5682. Min Act: -0.1700

Data Index: 604578 (Open Web Text)

Max Activating Token Index: 55

Text #8

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5856. Min Act: -0.1700

Data Index: 1455069 (Open Web Text)

Max Activating Token Index: 439

Text #9

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5261. Min Act: -0.1700

Data Index: 3907869 (Open Web Text)

Max Activating Token Index: 52

Text #10

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5257. Min Act: -0.1700

Data Index: 3557969 (Open Web Text)

Max Activating Token Index: 402

Text #11

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5521. Min Act: -0.1700

Data Index: 1567314 (Open Web Text)

Max Activating Token Index: 266

Text #12

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5366. Min Act: -0.1700

Data Index: 254281 (Open Web Text)

Max Activating Token Index: 388

Text #13

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5156. Min Act: -0.1700

Data Index: 2707351 (Open Web Text)

Max Activating Token Index: 255

Text #14

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5298. Min Act: -0.1700

Data Index: 2421963 (Open Web Text)

Max Activating Token Index: 516

Text #15

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5378. Min Act: -0.1700

Data Index: 6037526 (Open Web Text)

Max Activating Token Index: 361

Text #16

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5393. Min Act: -0.1700

Data Index: 5713271 (Open Web Text)

Max Activating Token Index: 22

Text #17

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.5007. Min Act: -0.1700

Data Index: 4295127 (Open Web Text)

Max Activating Token Index: 434

Text #18

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.4778. Min Act: -0.1700

Data Index: 1827857 (Open Web Text)

Max Activating Token Index: 948

Text #19

Max Range: 3.6537. Min Range: -3.6537

Max Act: 3.4745. Min Act: -0.1700

Data Index: 3792080 (Open Web Text)

Max Activating Token Index: 470