Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1040 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2134. Min Range: -4.2134

Max Act: 4.2134. Min Act: -0.1700

Data Index: 3854555 (Open Web Text)

Max Activating Token Index: 259

Text #1

Max Range: 4.2134. Min Range: -4.2134

Max Act: 4.0411. Min Act: -0.1700

Data Index: 7446274 (Open Web Text)

Max Activating Token Index: 450

Text #2

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8926. Min Act: -0.1700

Data Index: 7033431 (Open Web Text)

Max Activating Token Index: 693

Text #3

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8943. Min Act: -0.1700

Data Index: 6326099 (Open Web Text)

Max Activating Token Index: 948

Text #4

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8988. Min Act: -0.1700

Data Index: 3537409 (Open Web Text)

Max Activating Token Index: 205

Text #5

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.9399. Min Act: -0.1700

Data Index: 14340 (Open Web Text)

Max Activating Token Index: 578

Text #6

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.9107. Min Act: -0.1700

Data Index: 5198402 (Open Web Text)

Max Activating Token Index: 659

Text #7

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8856. Min Act: -0.1700

Data Index: 3910603 (Open Web Text)

Max Activating Token Index: 963

Text #8

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8951. Min Act: -0.1700

Data Index: 2079398 (Open Web Text)

Max Activating Token Index: 385

Text #9

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8851. Min Act: -0.1700

Data Index: 4161984 (Open Web Text)

Max Activating Token Index: 52

Text #10

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8494. Min Act: -0.1700

Data Index: 4167279 (Open Web Text)

Max Activating Token Index: 749

Text #11

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8450. Min Act: -0.1700

Data Index: 8387521 (Open Web Text)

Max Activating Token Index: 730

Text #12

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8731. Min Act: -0.1700

Data Index: 7138382 (Open Web Text)

Max Activating Token Index: 394

Text #13

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8826. Min Act: -0.1700

Data Index: 1123095 (Open Web Text)

Max Activating Token Index: 600

Text #14

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8375. Min Act: -0.1700

Data Index: 2225559 (Open Web Text)

Max Activating Token Index: 383

Text #15

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8175. Min Act: -0.1700

Data Index: 1460341 (Open Web Text)

Max Activating Token Index: 23

Text #16

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8108. Min Act: -0.1700

Data Index: 5342067 (Open Web Text)

Max Activating Token Index: 609

Text #17

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8419. Min Act: -0.1700

Data Index: 5769903 (Open Web Text)

Max Activating Token Index: 421

Text #18

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8202. Min Act: -0.1700

Data Index: 1000251 (Open Web Text)

Max Activating Token Index: 136

Text #19

Max Range: 4.2134. Min Range: -4.2134

Max Act: 3.8041. Min Act: -0.1700

Data Index: 6100212 (Open Web Text)

Max Activating Token Index: 683