Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 897 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.8449. Min Act: -0.1700

Data Index: 1244640 (Open Web Text)

Max Activating Token Index: 430

Text #1

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.7301. Min Act: -0.1700

Data Index: 6664524 (Open Web Text)

Max Activating Token Index: 615

Text #2

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.7171. Min Act: -0.1700

Data Index: 6568802 (Open Web Text)

Max Activating Token Index: 602

Text #3

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6815. Min Act: -0.1700

Data Index: 2640523 (Open Web Text)

Max Activating Token Index: 579

Text #4

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6995. Min Act: -0.1700

Data Index: 7549174 (Open Web Text)

Max Activating Token Index: 376

Text #5

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6961. Min Act: -0.1700

Data Index: 4440358 (Open Web Text)

Max Activating Token Index: 208

Text #6

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6582. Min Act: -0.1700

Data Index: 3780246 (Open Web Text)

Max Activating Token Index: 818

Text #7

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6766. Min Act: -0.1700

Data Index: 3337590 (Open Web Text)

Max Activating Token Index: 885

Text #8

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6338. Min Act: -0.1700

Data Index: 16980 (Open Web Text)

Max Activating Token Index: 450

Text #9

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6347. Min Act: -0.1700

Data Index: 4178234 (Open Web Text)

Max Activating Token Index: 535

Text #10

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6408. Min Act: -0.1700

Data Index: 5848729 (Open Web Text)

Max Activating Token Index: 722

Text #11

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6389. Min Act: -0.1700

Data Index: 5329634 (Open Web Text)

Max Activating Token Index: 85

Text #12

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6242. Min Act: -0.1700

Data Index: 382258 (Open Web Text)

Max Activating Token Index: 550

Text #13

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6320. Min Act: -0.1700

Data Index: 1501984 (Open Web Text)

Max Activating Token Index: 926

Text #14

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6412. Min Act: -0.1700

Data Index: 2429916 (Open Web Text)

Max Activating Token Index: 674

Text #15

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6149. Min Act: -0.1700

Data Index: 8185708 (Open Web Text)

Max Activating Token Index: 173

Text #16

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6101. Min Act: -0.1700

Data Index: 4319200 (Open Web Text)

Max Activating Token Index: 623

Text #17

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6243. Min Act: -0.1700

Data Index: 5698903 (Open Web Text)

Max Activating Token Index: 642

Text #18

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.6107. Min Act: -0.1700

Data Index: 4512868 (Open Web Text)

Max Activating Token Index: 536

Text #19

Max Range: 3.8449. Min Range: -3.8449

Max Act: 3.5975. Min Act: -0.1700

Data Index: 1290913 (Open Web Text)

Max Activating Token Index: 1022