Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 670 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.4169. Min Act: -0.1700

Data Index: 7029572 (Open Web Text)

Max Activating Token Index: 541

Text #1

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.3453. Min Act: -0.1700

Data Index: 4268608 (Open Web Text)

Max Activating Token Index: 728

Text #2

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2842. Min Act: -0.1700

Data Index: 7558517 (Open Web Text)

Max Activating Token Index: 33

Text #3

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2911. Min Act: -0.1700

Data Index: 8571507 (Open Web Text)

Max Activating Token Index: 99

Text #4

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2811. Min Act: -0.1700

Data Index: 5071013 (Open Web Text)

Max Activating Token Index: 44

Text #5

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2670. Min Act: -0.1700

Data Index: 257463 (Open Web Text)

Max Activating Token Index: 36

Text #6

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2434. Min Act: -0.1700

Data Index: 7578549 (Open Web Text)

Max Activating Token Index: 66

Text #7

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2455. Min Act: -0.1700

Data Index: 3589752 (Open Web Text)

Max Activating Token Index: 49

Text #8

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2431. Min Act: -0.1700

Data Index: 6906143 (Open Web Text)

Max Activating Token Index: 24

Text #9

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2343. Min Act: -0.1700

Data Index: 6205593 (Open Web Text)

Max Activating Token Index: 70

Text #10

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2243. Min Act: -0.1700

Data Index: 3851381 (Open Web Text)

Max Activating Token Index: 246

Text #11

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2377. Min Act: -0.1700

Data Index: 3820115 (Open Web Text)

Max Activating Token Index: 168

Text #12

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2323. Min Act: -0.1700

Data Index: 4495421 (Open Web Text)

Max Activating Token Index: 112

Text #13

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2362. Min Act: -0.1700

Data Index: 3622695 (Open Web Text)

Max Activating Token Index: 698

Text #14

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2168. Min Act: -0.1700

Data Index: 7334413 (Open Web Text)

Max Activating Token Index: 375

Text #15

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2070. Min Act: -0.1700

Data Index: 8084343 (Open Web Text)

Max Activating Token Index: 672

Text #16

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2044. Min Act: -0.1700

Data Index: 695194 (Open Web Text)

Max Activating Token Index: 384

Text #17

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2111. Min Act: -0.1700

Data Index: 8807983 (Open Web Text)

Max Activating Token Index: 456

Text #18

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.2023. Min Act: -0.1700

Data Index: 1192221 (Open Web Text)

Max Activating Token Index: 25

Text #19

Max Range: 3.4169. Min Range: -3.4169

Max Act: 3.1949. Min Act: -0.1700

Data Index: 3123435 (Open Web Text)

Max Activating Token Index: 135