Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2061 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.7852. Min Act: -0.1700

Data Index: 6231824 (Open Web Text)

Max Activating Token Index: 619

Text #1

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.6860. Min Act: -0.1700

Data Index: 3559697 (Open Web Text)

Max Activating Token Index: 462

Text #2

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.3395. Min Act: -0.1700

Data Index: 5394520 (Open Web Text)

Max Activating Token Index: 143

Text #3

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.2701. Min Act: -0.1700

Data Index: 921642 (Open Web Text)

Max Activating Token Index: 866

Text #4

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.2345. Min Act: -0.1700

Data Index: 3537285 (Open Web Text)

Max Activating Token Index: 386

Text #5

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.2182. Min Act: -0.1700

Data Index: 5248833 (Open Web Text)

Max Activating Token Index: 270

Text #6

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.0825. Min Act: -0.1700

Data Index: 447963 (Open Web Text)

Max Activating Token Index: 505

Text #7

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.0475. Min Act: -0.1700

Data Index: 4502075 (Open Web Text)

Max Activating Token Index: 587

Text #8

Max Range: 5.7852. Min Range: -5.7852

Max Act: 5.0072. Min Act: -0.1700

Data Index: 3559703 (Open Web Text)

Max Activating Token Index: 632

Text #9

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.9173. Min Act: -0.1700

Data Index: 3904176 (Open Web Text)

Max Activating Token Index: 609

Text #10

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8643. Min Act: -0.1700

Data Index: 7521213 (Open Web Text)

Max Activating Token Index: 457

Text #11

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8590. Min Act: -0.1700

Data Index: 3326286 (Open Web Text)

Max Activating Token Index: 960

Text #12

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8765. Min Act: -0.1700

Data Index: 7039996 (Open Web Text)

Max Activating Token Index: 801

Text #13

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8741. Min Act: -0.1700

Data Index: 7450931 (Open Web Text)

Max Activating Token Index: 112

Text #14

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8337. Min Act: -0.1700

Data Index: 7524618 (Open Web Text)

Max Activating Token Index: 1008

Text #15

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8470. Min Act: -0.1700

Data Index: 5895215 (Open Web Text)

Max Activating Token Index: 652

Text #16

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8464. Min Act: -0.1700

Data Index: 7535254 (Open Web Text)

Max Activating Token Index: 690

Text #17

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8552. Min Act: -0.1700

Data Index: 601418 (Open Web Text)

Max Activating Token Index: 649

Text #18

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8544. Min Act: -0.1700

Data Index: 6625868 (Open Web Text)

Max Activating Token Index: 933

Text #19

Max Range: 5.7852. Min Range: -5.7852

Max Act: 4.8126. Min Act: -0.1700

Data Index: 6991731 (Open Web Text)

Max Activating Token Index: 95