Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2146 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.7097. Min Act: -0.1695

Data Index: 4064370 (Open Web Text)

Max Activating Token Index: 545

Text #1

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.7013. Min Act: -0.1700

Data Index: 3112461 (Open Web Text)

Max Activating Token Index: 145

Text #2

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.6140. Min Act: -0.1700

Data Index: 7942577 (Open Web Text)

Max Activating Token Index: 369

Text #3

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5985. Min Act: -0.1700

Data Index: 6943952 (Open Web Text)

Max Activating Token Index: 355

Text #4

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5492. Min Act: -0.1700

Data Index: 7088768 (Open Web Text)

Max Activating Token Index: 113

Text #5

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.4589. Min Act: -0.1700

Data Index: 4787644 (Open Web Text)

Max Activating Token Index: 430

Text #6

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5128. Min Act: -0.1700

Data Index: 4544290 (Open Web Text)

Max Activating Token Index: 833

Text #7

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.4999. Min Act: -0.1700

Data Index: 6618070 (Open Web Text)

Max Activating Token Index: 47

Text #8

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5574. Min Act: -0.1700

Data Index: 876907 (Open Web Text)

Max Activating Token Index: 432

Text #9

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5452. Min Act: -0.1700

Data Index: 3240961 (Open Web Text)

Max Activating Token Index: 49

Text #10

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.6200. Min Act: -0.1700

Data Index: 5047047 (Open Web Text)

Max Activating Token Index: 690

Text #11

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5785. Min Act: -0.1698

Data Index: 1058823 (Open Web Text)

Max Activating Token Index: 652

Text #12

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.4876. Min Act: -0.1699

Data Index: 7045831 (Open Web Text)

Max Activating Token Index: 608

Text #13

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.4814. Min Act: -0.1700

Data Index: 4060703 (Open Web Text)

Max Activating Token Index: 632

Text #14

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5526. Min Act: -0.1694

Data Index: 2841494 (Open Web Text)

Max Activating Token Index: 82

Text #15

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5442. Min Act: -0.1700

Data Index: 3996474 (Open Web Text)

Max Activating Token Index: 662

Text #16

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.5189. Min Act: -0.1700

Data Index: 6847903 (Open Web Text)

Max Activating Token Index: 128

Text #17

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.4890. Min Act: -0.1700

Data Index: 270757 (Open Web Text)

Max Activating Token Index: 20

Text #18

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.4807. Min Act: -0.1700

Data Index: 2642460 (Open Web Text)

Max Activating Token Index: 106

Text #19

Max Range: 5.7097. Min Range: -5.7097

Max Act: 5.4728. Min Act: -0.1699

Data Index: 3575496 (Open Web Text)

Max Activating Token Index: 124