Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 972 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0461. Min Range: -2.0461

Max Act: 2.0461. Min Act: -0.1700

Data Index: 7197053 (Open Web Text)

Max Activating Token Index: 715

Text #1

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9910. Min Act: -0.1700

Data Index: 3803732 (Open Web Text)

Max Activating Token Index: 116

Text #2

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9799. Min Act: -0.1700

Data Index: 526503 (Open Web Text)

Max Activating Token Index: 137

Text #3

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9817. Min Act: -0.1699

Data Index: 5729216 (Open Web Text)

Max Activating Token Index: 285

Text #4

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9773. Min Act: -0.1700

Data Index: 3103761 (Open Web Text)

Max Activating Token Index: 270

Text #5

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9462. Min Act: -0.1700

Data Index: 2823368 (Open Web Text)

Max Activating Token Index: 786

Text #6

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9406. Min Act: -0.1700

Data Index: 8724273 (Open Web Text)

Max Activating Token Index: 104

Text #7

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9469. Min Act: -0.1700

Data Index: 1783759 (Open Web Text)

Max Activating Token Index: 698

Text #8

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9527. Min Act: -0.1700

Data Index: 6563754 (Open Web Text)

Max Activating Token Index: 177

Text #9

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9404. Min Act: -0.1700

Data Index: 2151999 (Open Web Text)

Max Activating Token Index: 455

Text #10

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9307. Min Act: -0.1700

Data Index: 5903413 (Open Web Text)

Max Activating Token Index: 90

Text #11

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9358. Min Act: -0.1700

Data Index: 7867895 (Open Web Text)

Max Activating Token Index: 717

Text #12

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9403. Min Act: -0.1700

Data Index: 3278938 (Open Web Text)

Max Activating Token Index: 65

Text #13

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9358. Min Act: -0.1700

Data Index: 7863504 (Open Web Text)

Max Activating Token Index: 656

Text #14

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9288. Min Act: -0.1700

Data Index: 5389022 (Open Web Text)

Max Activating Token Index: 688

Text #15

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9198. Min Act: -0.1700

Data Index: 1789688 (Open Web Text)

Max Activating Token Index: 317

Text #16

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9218. Min Act: -0.1700

Data Index: 3833884 (Open Web Text)

Max Activating Token Index: 105

Text #17

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9158. Min Act: -0.1700

Data Index: 4011535 (Open Web Text)

Max Activating Token Index: 473

Text #18

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9171. Min Act: -0.1700

Data Index: 7048852 (Open Web Text)

Max Activating Token Index: 372

Text #19

Max Range: 2.0461. Min Range: -2.0461

Max Act: 1.9225. Min Act: -0.1700

Data Index: 4684193 (Open Web Text)

Max Activating Token Index: 698