Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1606 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1987. Min Act: -0.1700

Data Index: 7318257 (Open Web Text)

Max Activating Token Index: 957

Text #1

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1665. Min Act: -0.1700

Data Index: 8496042 (Open Web Text)

Max Activating Token Index: 959

Text #2

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1566. Min Act: -0.1700

Data Index: 1564715 (Open Web Text)

Max Activating Token Index: 971

Text #3

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1495. Min Act: -0.1700

Data Index: 7085779 (Open Web Text)

Max Activating Token Index: 971

Text #4

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1534. Min Act: -0.1700

Data Index: 4345449 (Open Web Text)

Max Activating Token Index: 990

Text #5

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1402. Min Act: -0.1700

Data Index: 7518713 (Open Web Text)

Max Activating Token Index: 972

Text #6

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1330. Min Act: -0.1700

Data Index: 222080 (Open Web Text)

Max Activating Token Index: 971

Text #7

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1323. Min Act: -0.1700

Data Index: 699967 (Open Web Text)

Max Activating Token Index: 979

Text #8

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1344. Min Act: -0.1700

Data Index: 2812748 (Open Web Text)

Max Activating Token Index: 981

Text #9

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1366. Min Act: -0.1700

Data Index: 3938166 (Open Web Text)

Max Activating Token Index: 940

Text #10

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1229. Min Act: -0.1700

Data Index: 2539345 (Open Web Text)

Max Activating Token Index: 964

Text #11

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1249. Min Act: -0.1700

Data Index: 2102007 (Open Web Text)

Max Activating Token Index: 969

Text #12

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1259. Min Act: -0.1700

Data Index: 5806004 (Open Web Text)

Max Activating Token Index: 991

Text #13

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1283. Min Act: -0.1700

Data Index: 872628 (Open Web Text)

Max Activating Token Index: 976

Text #14

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1235. Min Act: -0.1700

Data Index: 7006210 (Open Web Text)

Max Activating Token Index: 960

Text #15

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1123. Min Act: -0.1700

Data Index: 1292082 (Open Web Text)

Max Activating Token Index: 978

Text #16

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1172. Min Act: -0.1700

Data Index: 6027080 (Open Web Text)

Max Activating Token Index: 957

Text #17

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1141. Min Act: -0.1700

Data Index: 6356394 (Open Web Text)

Max Activating Token Index: 962

Text #18

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1114. Min Act: -0.1700

Data Index: 1507389 (Open Web Text)

Max Activating Token Index: 978

Text #19

Max Range: 1.1987. Min Range: -1.1987

Max Act: 1.1221. Min Act: -0.1700

Data Index: 6851944 (Open Web Text)

Max Activating Token Index: 960