Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1685 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.8066. Min Act: -0.1700

Data Index: 6084991 (Open Web Text)

Max Activating Token Index: 231

Text #1

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.8555. Min Act: -0.1700

Data Index: 3187924 (Open Web Text)

Max Activating Token Index: 589

Text #2

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7963. Min Act: -0.1700

Data Index: 8783037 (Open Web Text)

Max Activating Token Index: 191

Text #3

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.8002. Min Act: -0.1700

Data Index: 6302302 (Open Web Text)

Max Activating Token Index: 883

Text #4

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7804. Min Act: -0.1700

Data Index: 8599554 (Open Web Text)

Max Activating Token Index: 516

Text #5

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7877. Min Act: -0.1700

Data Index: 4871044 (Open Web Text)

Max Activating Token Index: 233

Text #6

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7682. Min Act: -0.1700

Data Index: 6400549 (Open Web Text)

Max Activating Token Index: 390

Text #7

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7435. Min Act: -0.1700

Data Index: 3510739 (Open Web Text)

Max Activating Token Index: 119

Text #8

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7263. Min Act: -0.1700

Data Index: 8620288 (Open Web Text)

Max Activating Token Index: 321

Text #9

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7283. Min Act: -0.1700

Data Index: 2508085 (Open Web Text)

Max Activating Token Index: 628

Text #10

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7157. Min Act: -0.1700

Data Index: 7866078 (Open Web Text)

Max Activating Token Index: 647

Text #11

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.6981. Min Act: -0.1700

Data Index: 2145429 (Open Web Text)

Max Activating Token Index: 788

Text #12

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7159. Min Act: -0.1700

Data Index: 4494208 (Open Web Text)

Max Activating Token Index: 163

Text #13

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7043. Min Act: -0.1700

Data Index: 948102 (Open Web Text)

Max Activating Token Index: 253

Text #14

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.7049. Min Act: -0.1700

Data Index: 7520324 (Open Web Text)

Max Activating Token Index: 919

Text #15

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.6935. Min Act: -0.1700

Data Index: 3407143 (Open Web Text)

Max Activating Token Index: 626

Text #16

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.6623. Min Act: -0.1700

Data Index: 1632566 (Open Web Text)

Max Activating Token Index: 722

Text #17

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.6770. Min Act: -0.1700

Data Index: 3721868 (Open Web Text)

Max Activating Token Index: 594

Text #18

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.6723. Min Act: -0.1700

Data Index: 6902801 (Open Web Text)

Max Activating Token Index: 667

Text #19

Max Range: 5.8555. Min Range: -5.8555

Max Act: 5.6665. Min Act: -0.1700

Data Index: 2958397 (Open Web Text)

Max Activating Token Index: 92