Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1300 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.3080. Min Act: -0.1700

Data Index: 6356406 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2612. Min Act: -0.1700

Data Index: 8525062 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2349. Min Act: -0.1700

Data Index: 4385069 (Open Web Text)

Max Activating Token Index: 994

Text #3

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2246. Min Act: -0.1700

Data Index: 7350974 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2274. Min Act: -0.1700

Data Index: 5444169 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2208. Min Act: -0.1700

Data Index: 6572446 (Open Web Text)

Max Activating Token Index: 14

Text #6

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2468. Min Act: -0.1700

Data Index: 1508587 (Open Web Text)

Max Activating Token Index: 17

Text #7

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2287. Min Act: -0.1700

Data Index: 4066631 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2177. Min Act: -0.1700

Data Index: 1547631 (Open Web Text)

Max Activating Token Index: 13

Text #9

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.1916. Min Act: -0.1700

Data Index: 1565892 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.1943. Min Act: -0.1700

Data Index: 4357348 (Open Web Text)

Max Activating Token Index: 14

Text #11

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.1890. Min Act: -0.1700

Data Index: 7193342 (Open Web Text)

Max Activating Token Index: 1007

Text #12

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.1958. Min Act: -0.1700

Data Index: 307117 (Open Web Text)

Max Activating Token Index: 18

Text #13

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2026. Min Act: -0.1700

Data Index: 3678676 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2003. Min Act: -0.1700

Data Index: 5851747 (Open Web Text)

Max Activating Token Index: 11

Text #15

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2094. Min Act: -0.1700

Data Index: 8155792 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2201. Min Act: -0.1700

Data Index: 7192394 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.2107. Min Act: -0.1700

Data Index: 6362018 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.1882. Min Act: -0.1700

Data Index: 524712 (Open Web Text)

Max Activating Token Index: 8

Text #19

Max Range: 5.3080. Min Range: -5.3080

Max Act: 5.1850. Min Act: -0.1700

Data Index: 2219743 (Open Web Text)

Max Activating Token Index: 1015