Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 384 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.5462. Min Act: -0.1700

Data Index: 6735960 (Open Web Text)

Max Activating Token Index: 236

Text #1

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.5272. Min Act: -0.1700

Data Index: 3104015 (Open Web Text)

Max Activating Token Index: 906

Text #2

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.5697. Min Act: -0.1700

Data Index: 8103974 (Open Web Text)

Max Activating Token Index: 983

Text #3

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4780. Min Act: -0.1700

Data Index: 8283984 (Open Web Text)

Max Activating Token Index: 183

Text #4

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4634. Min Act: -0.1700

Data Index: 4239166 (Open Web Text)

Max Activating Token Index: 762

Text #5

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4585. Min Act: -0.1700

Data Index: 445647 (Open Web Text)

Max Activating Token Index: 923

Text #6

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4347. Min Act: -0.1700

Data Index: 8109706 (Open Web Text)

Max Activating Token Index: 151

Text #7

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4721. Min Act: -0.1700

Data Index: 3289602 (Open Web Text)

Max Activating Token Index: 378

Text #8

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4228. Min Act: -0.1700

Data Index: 6907176 (Open Web Text)

Max Activating Token Index: 409

Text #9

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4419. Min Act: -0.1700

Data Index: 2197924 (Open Web Text)

Max Activating Token Index: 748

Text #10

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3755. Min Act: -0.1700

Data Index: 2560951 (Open Web Text)

Max Activating Token Index: 688

Text #11

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.4046. Min Act: -0.1700

Data Index: 704982 (Open Web Text)

Max Activating Token Index: 67

Text #12

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3346. Min Act: -0.1700

Data Index: 3630401 (Open Web Text)

Max Activating Token Index: 292

Text #13

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3581. Min Act: -0.1700

Data Index: 7045752 (Open Web Text)

Max Activating Token Index: 727

Text #14

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3753. Min Act: -0.1700

Data Index: 67289 (Open Web Text)

Max Activating Token Index: 136

Text #15

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3536. Min Act: -0.1700

Data Index: 5802754 (Open Web Text)

Max Activating Token Index: 781

Text #16

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3514. Min Act: -0.1700

Data Index: 6969635 (Open Web Text)

Max Activating Token Index: 130

Text #17

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3434. Min Act: -0.1700

Data Index: 3560065 (Open Web Text)

Max Activating Token Index: 791

Text #18

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3636. Min Act: -0.1700

Data Index: 6257905 (Open Web Text)

Max Activating Token Index: 289

Text #19

Max Range: 5.5697. Min Range: -5.5697

Max Act: 5.3009. Min Act: -0.1700

Data Index: 2362362 (Open Web Text)

Max Activating Token Index: 812