Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 707 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.4743. Min Act: -0.1700

Data Index: 6257980 (Open Web Text)

Max Activating Token Index: 129

Text #1

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.4542. Min Act: -0.1700

Data Index: 5819243 (Open Web Text)

Max Activating Token Index: 113

Text #2

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3987. Min Act: -0.1700

Data Index: 4188415 (Open Web Text)

Max Activating Token Index: 121

Text #3

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3533. Min Act: -0.1700

Data Index: 752787 (Open Web Text)

Max Activating Token Index: 380

Text #4

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3534. Min Act: -0.1700

Data Index: 8062038 (Open Web Text)

Max Activating Token Index: 272

Text #5

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3467. Min Act: -0.1700

Data Index: 4008091 (Open Web Text)

Max Activating Token Index: 552

Text #6

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3322. Min Act: -0.1700

Data Index: 1384880 (Open Web Text)

Max Activating Token Index: 163

Text #7

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3289. Min Act: -0.1700

Data Index: 670716 (Open Web Text)

Max Activating Token Index: 126

Text #8

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3185. Min Act: -0.1700

Data Index: 5936457 (Open Web Text)

Max Activating Token Index: 151

Text #9

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3202. Min Act: -0.1700

Data Index: 4466253 (Open Web Text)

Max Activating Token Index: 200

Text #10

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3087. Min Act: -0.1700

Data Index: 5024331 (Open Web Text)

Max Activating Token Index: 238

Text #11

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3102. Min Act: -0.1700

Data Index: 6437724 (Open Web Text)

Max Activating Token Index: 140

Text #12

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3018. Min Act: -0.1700

Data Index: 1687694 (Open Web Text)

Max Activating Token Index: 306

Text #13

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3148. Min Act: -0.1700

Data Index: 1982285 (Open Web Text)

Max Activating Token Index: 198

Text #14

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.3012. Min Act: -0.1700

Data Index: 2827967 (Open Web Text)

Max Activating Token Index: 424

Text #15

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.2775. Min Act: -0.1700

Data Index: 8188466 (Open Web Text)

Max Activating Token Index: 125

Text #16

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.2807. Min Act: -0.1700

Data Index: 1926026 (Open Web Text)

Max Activating Token Index: 27

Text #17

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.2758. Min Act: -0.1700

Data Index: 2016941 (Open Web Text)

Max Activating Token Index: 391

Text #18

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.2586. Min Act: -0.1700

Data Index: 1100278 (Open Web Text)

Max Activating Token Index: 534

Text #19

Max Range: 3.4743. Min Range: -3.4743

Max Act: 3.2328. Min Act: -0.1700

Data Index: 5935352 (Open Web Text)

Max Activating Token Index: 547