Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 960 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.7393. Min Act: -0.1700

Data Index: 8686284 (Open Web Text)

Max Activating Token Index: 170

Text #1

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.7232. Min Act: -0.1700

Data Index: 6399162 (Open Web Text)

Max Activating Token Index: 402

Text #2

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.7266. Min Act: -0.1700

Data Index: 2946829 (Open Web Text)

Max Activating Token Index: 806

Text #3

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6821. Min Act: -0.1700

Data Index: 2898094 (Open Web Text)

Max Activating Token Index: 11

Text #4

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6638. Min Act: -0.1700

Data Index: 818009 (Open Web Text)

Max Activating Token Index: 466

Text #5

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6488. Min Act: -0.1700

Data Index: 2377768 (Open Web Text)

Max Activating Token Index: 454

Text #6

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6595. Min Act: -0.1700

Data Index: 1412167 (Open Web Text)

Max Activating Token Index: 528

Text #7

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6423. Min Act: -0.1700

Data Index: 4781756 (Open Web Text)

Max Activating Token Index: 135

Text #8

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6397. Min Act: -0.1700

Data Index: 3668 (Open Web Text)

Max Activating Token Index: 704

Text #9

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6258. Min Act: -0.1700

Data Index: 1343407 (Open Web Text)

Max Activating Token Index: 608

Text #10

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6148. Min Act: -0.1700

Data Index: 7744327 (Open Web Text)

Max Activating Token Index: 13

Text #11

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6256. Min Act: -0.1700

Data Index: 8552639 (Open Web Text)

Max Activating Token Index: 998

Text #12

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6265. Min Act: -0.1700

Data Index: 8402668 (Open Web Text)

Max Activating Token Index: 663

Text #13

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6467. Min Act: -0.1700

Data Index: 372262 (Open Web Text)

Max Activating Token Index: 581

Text #14

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6217. Min Act: -0.1700

Data Index: 1436594 (Open Web Text)

Max Activating Token Index: 435

Text #15

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6224. Min Act: -0.1700

Data Index: 7826570 (Open Web Text)

Max Activating Token Index: 935

Text #16

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6197. Min Act: -0.1700

Data Index: 63181 (Open Web Text)

Max Activating Token Index: 458

Text #17

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6091. Min Act: -0.1700

Data Index: 7380097 (Open Web Text)

Max Activating Token Index: 563

Text #18

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6109. Min Act: -0.1700

Data Index: 37625 (Open Web Text)

Max Activating Token Index: 374

Text #19

Max Range: 3.7393. Min Range: -3.7393

Max Act: 3.6126. Min Act: -0.1700

Data Index: 980429 (Open Web Text)

Max Activating Token Index: 19