Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 710 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.3846. Min Act: -0.1700

Data Index: 2636554 (Open Web Text)

Max Activating Token Index: 465

Text #1

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.3862. Min Act: -0.1700

Data Index: 6062318 (Open Web Text)

Max Activating Token Index: 348

Text #2

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.3069. Min Act: -0.1700

Data Index: 1008983 (Open Web Text)

Max Activating Token Index: 738

Text #3

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.3070. Min Act: -0.1700

Data Index: 5025261 (Open Web Text)

Max Activating Token Index: 277

Text #4

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.3073. Min Act: -0.1700

Data Index: 5434868 (Open Web Text)

Max Activating Token Index: 725

Text #5

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.3013. Min Act: -0.1700

Data Index: 7761532 (Open Web Text)

Max Activating Token Index: 140

Text #6

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.3189. Min Act: -0.1700

Data Index: 4030669 (Open Web Text)

Max Activating Token Index: 914

Text #7

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2746. Min Act: -0.1700

Data Index: 7933071 (Open Web Text)

Max Activating Token Index: 45

Text #8

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2751. Min Act: -0.1700

Data Index: 7851414 (Open Web Text)

Max Activating Token Index: 597

Text #9

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2674. Min Act: -0.1700

Data Index: 2109609 (Open Web Text)

Max Activating Token Index: 284

Text #10

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2611. Min Act: -0.1700

Data Index: 4414550 (Open Web Text)

Max Activating Token Index: 811

Text #11

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2593. Min Act: -0.1700

Data Index: 5018695 (Open Web Text)

Max Activating Token Index: 860

Text #12

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2687. Min Act: -0.1700

Data Index: 5455516 (Open Web Text)

Max Activating Token Index: 806

Text #13

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2469. Min Act: -0.1700

Data Index: 725371 (Open Web Text)

Max Activating Token Index: 792

Text #14

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2165. Min Act: -0.1700

Data Index: 459388 (Open Web Text)

Max Activating Token Index: 923

Text #15

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2025. Min Act: -0.1700

Data Index: 106580 (Open Web Text)

Max Activating Token Index: 477

Text #16

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2104. Min Act: -0.1700

Data Index: 7787280 (Open Web Text)

Max Activating Token Index: 336

Text #17

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2225. Min Act: -0.1700

Data Index: 2961137 (Open Web Text)

Max Activating Token Index: 823

Text #18

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2070. Min Act: -0.1700

Data Index: 6594331 (Open Web Text)

Max Activating Token Index: 379

Text #19

Max Range: 3.3862. Min Range: -3.3862

Max Act: 3.2175. Min Act: -0.1700

Data Index: 61608 (Open Web Text)

Max Activating Token Index: 448