Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2585 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.0890. Min Range: -1.0890

Max Act: 1.0890. Min Act: -0.1700

Data Index: 6654274 (Open Web Text)

Max Activating Token Index: 620

Text #1

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.9147. Min Act: -0.1700

Data Index: 2688228 (Open Web Text)

Max Activating Token Index: 707

Text #2

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.9055. Min Act: -0.1700

Data Index: 2239506 (Open Web Text)

Max Activating Token Index: 63

Text #3

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8999. Min Act: -0.1700

Data Index: 5276810 (Open Web Text)

Max Activating Token Index: 64

Text #4

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8736. Min Act: -0.1588

Data Index: 5105989 (Open Web Text)

Max Activating Token Index: 887

Text #5

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8653. Min Act: -0.1682

Data Index: 2234140 (Open Web Text)

Max Activating Token Index: 360

Text #6

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8587. Min Act: -0.1487

Data Index: 5640266 (Open Web Text)

Max Activating Token Index: 196

Text #7

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8610. Min Act: -0.1688

Data Index: 1031022 (Open Web Text)

Max Activating Token Index: 189

Text #8

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8338. Min Act: -0.1597

Data Index: 2682281 (Open Web Text)

Max Activating Token Index: 620

Text #9

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8473. Min Act: -0.1681

Data Index: 4961027 (Open Web Text)

Max Activating Token Index: 387

Text #10

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8473. Min Act: -0.1689

Data Index: 6453116 (Open Web Text)

Max Activating Token Index: 1021

Text #11

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8505. Min Act: -0.1700

Data Index: 4611343 (Open Web Text)

Max Activating Token Index: 538

Text #12

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8406. Min Act: -0.1700

Data Index: 8716837 (Open Web Text)

Max Activating Token Index: 223

Text #13

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8305. Min Act: -0.1689

Data Index: 4745460 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8331. Min Act: -0.1697

Data Index: 5180844 (Open Web Text)

Max Activating Token Index: 456

Text #15

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8394. Min Act: -0.1616

Data Index: 5672346 (Open Web Text)

Max Activating Token Index: 633

Text #16

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8271. Min Act: -0.1644

Data Index: 646528 (Open Web Text)

Max Activating Token Index: 672

Text #17

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8243. Min Act: -0.1700

Data Index: 111178 (Open Web Text)

Max Activating Token Index: 74

Text #18

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8303. Min Act: -0.1608

Data Index: 6627681 (Open Web Text)

Max Activating Token Index: 962

Text #19

Max Range: 1.0890. Min Range: -1.0890

Max Act: 0.8178. Min Act: -0.1700

Data Index: 2902485 (Open Web Text)

Max Activating Token Index: 822