Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1213 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.9167. Min Act: -0.1700

Data Index: 8549492 (Open Web Text)

Max Activating Token Index: 676

Text #1

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.8970. Min Act: -0.1700

Data Index: 7292794 (Open Web Text)

Max Activating Token Index: 474

Text #2

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.8422. Min Act: -0.1700

Data Index: 8038013 (Open Web Text)

Max Activating Token Index: 212

Text #3

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.8010. Min Act: -0.1700

Data Index: 1059183 (Open Web Text)

Max Activating Token Index: 185

Text #4

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.8106. Min Act: -0.1700

Data Index: 4040768 (Open Web Text)

Max Activating Token Index: 417

Text #5

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.8091. Min Act: -0.1700

Data Index: 1020190 (Open Web Text)

Max Activating Token Index: 790

Text #6

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.8013. Min Act: -0.1700

Data Index: 1006024 (Open Web Text)

Max Activating Token Index: 748

Text #7

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.8015. Min Act: -0.1700

Data Index: 3995833 (Open Web Text)

Max Activating Token Index: 746

Text #8

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7649. Min Act: -0.1700

Data Index: 5356460 (Open Web Text)

Max Activating Token Index: 787

Text #9

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7783. Min Act: -0.1700

Data Index: 376136 (Open Web Text)

Max Activating Token Index: 264

Text #10

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7881. Min Act: -0.1700

Data Index: 5660071 (Open Web Text)

Max Activating Token Index: 491

Text #11

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7828. Min Act: -0.1700

Data Index: 6321853 (Open Web Text)

Max Activating Token Index: 636

Text #12

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7671. Min Act: -0.1700

Data Index: 5363866 (Open Web Text)

Max Activating Token Index: 283

Text #13

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7736. Min Act: -0.1700

Data Index: 6688109 (Open Web Text)

Max Activating Token Index: 688

Text #14

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7570. Min Act: -0.1700

Data Index: 7083130 (Open Web Text)

Max Activating Token Index: 588

Text #15

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7318. Min Act: -0.1700

Data Index: 7261355 (Open Web Text)

Max Activating Token Index: 915

Text #16

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7292. Min Act: -0.1700

Data Index: 2742891 (Open Web Text)

Max Activating Token Index: 877

Text #17

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.7273. Min Act: -0.1700

Data Index: 6208079 (Open Web Text)

Max Activating Token Index: 285

Text #18

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.6952. Min Act: -0.1700

Data Index: 5049651 (Open Web Text)

Max Activating Token Index: 58

Text #19

Max Range: 3.9167. Min Range: -3.9167

Max Act: 3.6842. Min Act: -0.1700

Data Index: 8549487 (Open Web Text)

Max Activating Token Index: 115