Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 573 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.5208. Min Act: -0.1700

Data Index: 6868617 (Open Web Text)

Max Activating Token Index: 44

Text #1

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.4386. Min Act: -0.1700

Data Index: 4158352 (Open Web Text)

Max Activating Token Index: 69

Text #2

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.4163. Min Act: -0.1700

Data Index: 1028863 (Open Web Text)

Max Activating Token Index: 14

Text #3

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.4103. Min Act: -0.1700

Data Index: 1183403 (Open Web Text)

Max Activating Token Index: 336

Text #4

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3935. Min Act: -0.1700

Data Index: 3170557 (Open Web Text)

Max Activating Token Index: 264

Text #5

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3973. Min Act: -0.1700

Data Index: 1598836 (Open Web Text)

Max Activating Token Index: 67

Text #6

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3684. Min Act: -0.1700

Data Index: 4637318 (Open Web Text)

Max Activating Token Index: 266

Text #7

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3674. Min Act: -0.1700

Data Index: 1252843 (Open Web Text)

Max Activating Token Index: 121

Text #8

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3629. Min Act: -0.1700

Data Index: 6984062 (Open Web Text)

Max Activating Token Index: 377

Text #9

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3186. Min Act: -0.1700

Data Index: 5458807 (Open Web Text)

Max Activating Token Index: 29

Text #10

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3119. Min Act: -0.1700

Data Index: 5097624 (Open Web Text)

Max Activating Token Index: 429

Text #11

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3253. Min Act: -0.1700

Data Index: 7875509 (Open Web Text)

Max Activating Token Index: 934

Text #12

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3066. Min Act: -0.1700

Data Index: 6555674 (Open Web Text)

Max Activating Token Index: 264

Text #13

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.3213. Min Act: -0.1700

Data Index: 818407 (Open Web Text)

Max Activating Token Index: 794

Text #14

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.2960. Min Act: -0.1700

Data Index: 6874841 (Open Web Text)

Max Activating Token Index: 214

Text #15

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.2970. Min Act: -0.1700

Data Index: 4514011 (Open Web Text)

Max Activating Token Index: 729

Text #16

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.2931. Min Act: -0.1700

Data Index: 6504731 (Open Web Text)

Max Activating Token Index: 373

Text #17

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.2857. Min Act: -0.1700

Data Index: 2467904 (Open Web Text)

Max Activating Token Index: 435

Text #18

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.2888. Min Act: -0.1700

Data Index: 6474982 (Open Web Text)

Max Activating Token Index: 61

Text #19

Max Range: 3.5208. Min Range: -3.5208

Max Act: 3.2881. Min Act: -0.1700

Data Index: 7313623 (Open Web Text)

Max Activating Token Index: 466