Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1609 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.9061. Min Act: -0.1700

Data Index: 5507943 (Open Web Text)

Max Activating Token Index: 477

Text #1

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.5078. Min Act: -0.1700

Data Index: 2878474 (Open Web Text)

Max Activating Token Index: 246

Text #2

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.4544. Min Act: -0.1700

Data Index: 6488140 (Open Web Text)

Max Activating Token Index: 762

Text #3

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.3835. Min Act: -0.1700

Data Index: 4272616 (Open Web Text)

Max Activating Token Index: 365

Text #4

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.3658. Min Act: -0.1700

Data Index: 3988289 (Open Web Text)

Max Activating Token Index: 378

Text #5

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.3479. Min Act: -0.1700

Data Index: 4751999 (Open Web Text)

Max Activating Token Index: 851

Text #6

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2909. Min Act: -0.1700

Data Index: 5566298 (Open Web Text)

Max Activating Token Index: 277

Text #7

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2904. Min Act: -0.1700

Data Index: 2536272 (Open Web Text)

Max Activating Token Index: 839

Text #8

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2861. Min Act: -0.1700

Data Index: 6854346 (Open Web Text)

Max Activating Token Index: 84

Text #9

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.3027. Min Act: -0.1700

Data Index: 6578981 (Open Web Text)

Max Activating Token Index: 408

Text #10

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2885. Min Act: -0.1700

Data Index: 1355565 (Open Web Text)

Max Activating Token Index: 421

Text #11

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.3290. Min Act: -0.1700

Data Index: 112573 (Open Web Text)

Max Activating Token Index: 618

Text #12

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2399. Min Act: -0.1700

Data Index: 8659173 (Open Web Text)

Max Activating Token Index: 887

Text #13

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2434. Min Act: -0.1700

Data Index: 7647879 (Open Web Text)

Max Activating Token Index: 259

Text #14

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2759. Min Act: -0.1700

Data Index: 1440582 (Open Web Text)

Max Activating Token Index: 210

Text #15

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2676. Min Act: -0.1700

Data Index: 407615 (Open Web Text)

Max Activating Token Index: 974

Text #16

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2503. Min Act: -0.1700

Data Index: 7682485 (Open Web Text)

Max Activating Token Index: 929

Text #17

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2982. Min Act: -0.1700

Data Index: 6529651 (Open Web Text)

Max Activating Token Index: 680

Text #18

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2471. Min Act: -0.1700

Data Index: 3023370 (Open Web Text)

Max Activating Token Index: 648

Text #19

Max Range: 4.9061. Min Range: -4.9061

Max Act: 4.2320. Min Act: -0.1700

Data Index: 3750722 (Open Web Text)

Max Activating Token Index: 649