Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1228 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.7097. Min Act: -0.1322

Data Index: 1709214 (Open Web Text)

Max Activating Token Index: 159

Text #1

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.7086. Min Act: -0.1067

Data Index: 6233662 (Open Web Text)

Max Activating Token Index: 316

Text #2

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6956. Min Act: -0.0541

Data Index: 5620903 (Open Web Text)

Max Activating Token Index: 882

Text #3

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6769. Min Act: -0.0680

Data Index: 2452379 (Open Web Text)

Max Activating Token Index: 564

Text #4

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6785. Min Act: -0.1457

Data Index: 2870285 (Open Web Text)

Max Activating Token Index: 483

Text #5

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6817. Min Act: -0.1047

Data Index: 2112772 (Open Web Text)

Max Activating Token Index: 513

Text #6

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6748. Min Act: -0.0795

Data Index: 2569695 (Open Web Text)

Max Activating Token Index: 326

Text #7

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6846. Min Act: -0.1065

Data Index: 6741588 (Open Web Text)

Max Activating Token Index: 959

Text #8

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6814. Min Act: -0.1504

Data Index: 8571623 (Open Web Text)

Max Activating Token Index: 483

Text #9

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6723. Min Act: -0.0988

Data Index: 470403 (Open Web Text)

Max Activating Token Index: 710

Text #10

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6742. Min Act: -0.1354

Data Index: 3897119 (Open Web Text)

Max Activating Token Index: 231

Text #11

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6732. Min Act: -0.0791

Data Index: 3335564 (Open Web Text)

Max Activating Token Index: 865

Text #12

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6638. Min Act: -0.1360

Data Index: 2399204 (Open Web Text)

Max Activating Token Index: 237

Text #13

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6605. Min Act: -0.1446

Data Index: 8435573 (Open Web Text)

Max Activating Token Index: 108

Text #14

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6614. Min Act: -0.1382

Data Index: 7758757 (Open Web Text)

Max Activating Token Index: 901

Text #15

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6668. Min Act: -0.1683

Data Index: 7597354 (Open Web Text)

Max Activating Token Index: 267

Text #16

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6553. Min Act: -0.0842

Data Index: 5034349 (Open Web Text)

Max Activating Token Index: 973

Text #17

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6548. Min Act: -0.0864

Data Index: 5493357 (Open Web Text)

Max Activating Token Index: 290

Text #18

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6461. Min Act: -0.0624

Data Index: 1163478 (Open Web Text)

Max Activating Token Index: 637

Text #19

Max Range: 1.7097. Min Range: -1.7097

Max Act: 1.6509. Min Act: -0.1369

Data Index: 1448684 (Open Web Text)

Max Activating Token Index: 519