Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 199 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.6794. Min Act: -0.1700

Data Index: 4824176 (Open Web Text)

Max Activating Token Index: 656

Text #1

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.6984. Min Act: -0.1700

Data Index: 96553 (Open Web Text)

Max Activating Token Index: 834

Text #2

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.6334. Min Act: -0.1700

Data Index: 3751266 (Open Web Text)

Max Activating Token Index: 86

Text #3

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.5702. Min Act: -0.1700

Data Index: 6295449 (Open Web Text)

Max Activating Token Index: 616

Text #4

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.5351. Min Act: -0.1700

Data Index: 3670626 (Open Web Text)

Max Activating Token Index: 696

Text #5

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.5475. Min Act: -0.1700

Data Index: 8239401 (Open Web Text)

Max Activating Token Index: 500

Text #6

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.3847. Min Act: -0.1700

Data Index: 2566108 (Open Web Text)

Max Activating Token Index: 545

Text #7

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.3824. Min Act: -0.1700

Data Index: 3414251 (Open Web Text)

Max Activating Token Index: 656

Text #8

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.3026. Min Act: -0.1700

Data Index: 6971118 (Open Web Text)

Max Activating Token Index: 746

Text #9

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.3097. Min Act: -0.1700

Data Index: 3800589 (Open Web Text)

Max Activating Token Index: 573

Text #10

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.2769. Min Act: -0.1700

Data Index: 3267393 (Open Web Text)

Max Activating Token Index: 512

Text #11

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.2242. Min Act: -0.1700

Data Index: 1864452 (Open Web Text)

Max Activating Token Index: 918

Text #12

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.3102. Min Act: -0.1700

Data Index: 5734766 (Open Web Text)

Max Activating Token Index: 919

Text #13

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.2414. Min Act: -0.1700

Data Index: 915107 (Open Web Text)

Max Activating Token Index: 231

Text #14

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.1925. Min Act: -0.1700

Data Index: 4710348 (Open Web Text)

Max Activating Token Index: 629

Text #15

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.2486. Min Act: -0.1700

Data Index: 5812298 (Open Web Text)

Max Activating Token Index: 812

Text #16

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.2216. Min Act: -0.1700

Data Index: 7760334 (Open Web Text)

Max Activating Token Index: 335

Text #17

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.2640. Min Act: -0.1700

Data Index: 5432116 (Open Web Text)

Max Activating Token Index: 660

Text #18

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.2141. Min Act: -0.1700

Data Index: 6799830 (Open Web Text)

Max Activating Token Index: 478

Text #19

Max Range: 6.6984. Min Range: -6.6984

Max Act: 6.1993. Min Act: -0.1700

Data Index: 1475586 (Open Web Text)

Max Activating Token Index: 442