Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 729 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.5146. Min Act: -0.1700

Data Index: 3434630 (Open Web Text)

Max Activating Token Index: 49

Text #1

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.4299. Min Act: -0.1700

Data Index: 3147989 (Open Web Text)

Max Activating Token Index: 588

Text #2

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.3683. Min Act: -0.1700

Data Index: 2730842 (Open Web Text)

Max Activating Token Index: 532

Text #3

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.3609. Min Act: -0.1700

Data Index: 7971879 (Open Web Text)

Max Activating Token Index: 223

Text #4

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.4077. Min Act: -0.1700

Data Index: 7062290 (Open Web Text)

Max Activating Token Index: 515

Text #5

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.3256. Min Act: -0.1700

Data Index: 2606693 (Open Web Text)

Max Activating Token Index: 687

Text #6

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.3399. Min Act: -0.1700

Data Index: 4327676 (Open Web Text)

Max Activating Token Index: 500

Text #7

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.3812. Min Act: -0.1700

Data Index: 3358285 (Open Web Text)

Max Activating Token Index: 680

Text #8

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2775. Min Act: -0.1700

Data Index: 5027169 (Open Web Text)

Max Activating Token Index: 727

Text #9

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2858. Min Act: -0.1700

Data Index: 2680776 (Open Web Text)

Max Activating Token Index: 997

Text #10

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2880. Min Act: -0.1700

Data Index: 1871529 (Open Web Text)

Max Activating Token Index: 845

Text #11

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.3023. Min Act: -0.1700

Data Index: 6214022 (Open Web Text)

Max Activating Token Index: 437

Text #12

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.3397. Min Act: -0.1700

Data Index: 2081202 (Open Web Text)

Max Activating Token Index: 50

Text #13

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2752. Min Act: -0.1700

Data Index: 684387 (Open Web Text)

Max Activating Token Index: 958

Text #14

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2519. Min Act: -0.1700

Data Index: 4085433 (Open Web Text)

Max Activating Token Index: 90

Text #15

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2601. Min Act: -0.1700

Data Index: 7420386 (Open Web Text)

Max Activating Token Index: 934

Text #16

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2641. Min Act: -0.1700

Data Index: 6402210 (Open Web Text)

Max Activating Token Index: 154

Text #17

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2475. Min Act: -0.1700

Data Index: 3217330 (Open Web Text)

Max Activating Token Index: 453

Text #18

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2699. Min Act: -0.1700

Data Index: 1734367 (Open Web Text)

Max Activating Token Index: 712

Text #19

Max Range: 4.5146. Min Range: -4.5146

Max Act: 4.2714. Min Act: -0.1700

Data Index: 6207855 (Open Web Text)

Max Activating Token Index: 484