Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1143 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2888. Min Act: -0.0354

Data Index: 690862 (Open Web Text)

Max Activating Token Index: 21

Text #1

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2687. Min Act: -0.0565

Data Index: 5385322 (Open Web Text)

Max Activating Token Index: 26

Text #2

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2664. Min Act: -0.0582

Data Index: 8806557 (Open Web Text)

Max Activating Token Index: 1021

Text #3

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2510. Min Act: -0.0640

Data Index: 8236894 (Open Web Text)

Max Activating Token Index: 1022

Text #4

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2481. Min Act: -0.0595

Data Index: 1306852 (Open Web Text)

Max Activating Token Index: 1022

Text #5

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2479. Min Act: -0.1074

Data Index: 7642828 (Open Web Text)

Max Activating Token Index: 33

Text #6

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2488. Min Act: -0.0636

Data Index: 7850149 (Open Web Text)

Max Activating Token Index: 34

Text #7

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2496. Min Act: -0.0603

Data Index: 2532945 (Open Web Text)

Max Activating Token Index: 1019

Text #8

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2443. Min Act: -0.0258

Data Index: 275704 (Open Web Text)

Max Activating Token Index: 45

Text #9

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2439. Min Act: -0.0100

Data Index: 6673929 (Open Web Text)

Max Activating Token Index: 53

Text #10

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2377. Min Act: -0.0711

Data Index: 2389500 (Open Web Text)

Max Activating Token Index: 1012

Text #11

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2351. Min Act: -0.0406

Data Index: 7601092 (Open Web Text)

Max Activating Token Index: 21

Text #12

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2277. Min Act: -0.0531

Data Index: 5990759 (Open Web Text)

Max Activating Token Index: 1021

Text #13

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2288. Min Act: -0.0218

Data Index: 4184437 (Open Web Text)

Max Activating Token Index: 1022

Text #14

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2298. Min Act: -0.0453

Data Index: 1230431 (Open Web Text)

Max Activating Token Index: 1017

Text #15

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2274. Min Act: -0.0623

Data Index: 6275966 (Open Web Text)

Max Activating Token Index: 173

Text #16

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2266. Min Act: -0.0506

Data Index: 960097 (Open Web Text)

Max Activating Token Index: 1021

Text #17

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2300. Min Act: -0.0319

Data Index: 6615621 (Open Web Text)

Max Activating Token Index: 1022

Text #18

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2297. Min Act: -0.0797

Data Index: 6718287 (Open Web Text)

Max Activating Token Index: 56

Text #19

Max Range: 1.2888. Min Range: -1.2888

Max Act: 1.2256. Min Act: -0.0589

Data Index: 2652444 (Open Web Text)

Max Activating Token Index: 19