Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 496 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.4128. Min Act: -0.1700

Data Index: 7808577 (Open Web Text)

Max Activating Token Index: 270

Text #1

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.2482. Min Act: -0.1700

Data Index: 2722870 (Open Web Text)

Max Activating Token Index: 233

Text #2

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.2589. Min Act: -0.1700

Data Index: 7348294 (Open Web Text)

Max Activating Token Index: 403

Text #3

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.2347. Min Act: -0.1700

Data Index: 3354361 (Open Web Text)

Max Activating Token Index: 534

Text #4

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.1687. Min Act: -0.1700

Data Index: 4646656 (Open Web Text)

Max Activating Token Index: 199

Text #5

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.1373. Min Act: -0.1700

Data Index: 5624349 (Open Web Text)

Max Activating Token Index: 922

Text #6

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.1088. Min Act: -0.1700

Data Index: 906601 (Open Web Text)

Max Activating Token Index: 801

Text #7

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.1415. Min Act: -0.1700

Data Index: 8193022 (Open Web Text)

Max Activating Token Index: 325

Text #8

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.1111. Min Act: -0.1700

Data Index: 7835571 (Open Web Text)

Max Activating Token Index: 506

Text #9

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.0689. Min Act: -0.1700

Data Index: 2375078 (Open Web Text)

Max Activating Token Index: 507

Text #10

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.0677. Min Act: -0.1700

Data Index: 4097503 (Open Web Text)

Max Activating Token Index: 405

Text #11

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.0329. Min Act: -0.1700

Data Index: 5412661 (Open Web Text)

Max Activating Token Index: 300

Text #12

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.0371. Min Act: -0.1700

Data Index: 4889112 (Open Web Text)

Max Activating Token Index: 598

Text #13

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.0354. Min Act: -0.1700

Data Index: 8431013 (Open Web Text)

Max Activating Token Index: 74

Text #14

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.0118. Min Act: -0.1700

Data Index: 5836650 (Open Web Text)

Max Activating Token Index: 250

Text #15

Max Range: 3.4128. Min Range: -3.4128

Max Act: 2.9862. Min Act: -0.1700

Data Index: 781887 (Open Web Text)

Max Activating Token Index: 87

Text #16

Max Range: 3.4128. Min Range: -3.4128

Max Act: 3.0077. Min Act: -0.1700

Data Index: 6747122 (Open Web Text)

Max Activating Token Index: 744

Text #17

Max Range: 3.4128. Min Range: -3.4128

Max Act: 2.9991. Min Act: -0.1700

Data Index: 4829656 (Open Web Text)

Max Activating Token Index: 629

Text #18

Max Range: 3.4128. Min Range: -3.4128

Max Act: 2.9486. Min Act: -0.1700

Data Index: 7379986 (Open Web Text)

Max Activating Token Index: 102

Text #19

Max Range: 3.4128. Min Range: -3.4128

Max Act: 2.9722. Min Act: -0.1700

Data Index: 505376 (Open Web Text)

Max Activating Token Index: 828