Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 425 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1015. Min Range: -4.1015

Max Act: 4.1015. Min Act: -0.1700

Data Index: 4821229 (Open Web Text)

Max Activating Token Index: 71

Text #1

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.9828. Min Act: -0.1700

Data Index: 3220060 (Open Web Text)

Max Activating Token Index: 356

Text #2

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.9685. Min Act: -0.1700

Data Index: 1950330 (Open Web Text)

Max Activating Token Index: 361

Text #3

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.9806. Min Act: -0.1700

Data Index: 2964441 (Open Web Text)

Max Activating Token Index: 355

Text #4

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.8898. Min Act: -0.1700

Data Index: 5604178 (Open Web Text)

Max Activating Token Index: 53

Text #5

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.8274. Min Act: -0.1700

Data Index: 7474189 (Open Web Text)

Max Activating Token Index: 132

Text #6

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.7314. Min Act: -0.1700

Data Index: 3459795 (Open Web Text)

Max Activating Token Index: 497

Text #7

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.7083. Min Act: -0.1700

Data Index: 3481146 (Open Web Text)

Max Activating Token Index: 93

Text #8

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.6958. Min Act: -0.1700

Data Index: 3315396 (Open Web Text)

Max Activating Token Index: 584

Text #9

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.6484. Min Act: -0.1700

Data Index: 6232048 (Open Web Text)

Max Activating Token Index: 50

Text #10

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5985. Min Act: -0.1700

Data Index: 2211284 (Open Web Text)

Max Activating Token Index: 611

Text #11

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5887. Min Act: -0.1700

Data Index: 3933161 (Open Web Text)

Max Activating Token Index: 743

Text #12

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5816. Min Act: -0.1700

Data Index: 6110096 (Open Web Text)

Max Activating Token Index: 977

Text #13

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5499. Min Act: -0.1700

Data Index: 1310993 (Open Web Text)

Max Activating Token Index: 386

Text #14

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5416. Min Act: -0.1700

Data Index: 170745 (Open Web Text)

Max Activating Token Index: 607

Text #15

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5389. Min Act: -0.1700

Data Index: 34035 (Open Web Text)

Max Activating Token Index: 882

Text #16

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5449. Min Act: -0.1700

Data Index: 3258525 (Open Web Text)

Max Activating Token Index: 133

Text #17

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5409. Min Act: -0.1700

Data Index: 5432832 (Open Web Text)

Max Activating Token Index: 301

Text #18

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.5487. Min Act: -0.1700

Data Index: 8489064 (Open Web Text)

Max Activating Token Index: 790

Text #19

Max Range: 4.1015. Min Range: -4.1015

Max Act: 3.4984. Min Act: -0.1700

Data Index: 4320967 (Open Web Text)

Max Activating Token Index: 903