Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 629 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.5753. Min Act: -0.1700

Data Index: 2986504 (Open Web Text)

Max Activating Token Index: 362

Text #1

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.5158. Min Act: -0.1700

Data Index: 8679828 (Open Web Text)

Max Activating Token Index: 966

Text #2

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.4087. Min Act: -0.1700

Data Index: 3591811 (Open Web Text)

Max Activating Token Index: 148

Text #3

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.3639. Min Act: -0.1700

Data Index: 5472640 (Open Web Text)

Max Activating Token Index: 800

Text #4

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.3241. Min Act: -0.1700

Data Index: 2125872 (Open Web Text)

Max Activating Token Index: 62

Text #5

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.3140. Min Act: -0.1700

Data Index: 6496861 (Open Web Text)

Max Activating Token Index: 271

Text #6

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.2300. Min Act: -0.1700

Data Index: 5726876 (Open Web Text)

Max Activating Token Index: 588

Text #7

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.2269. Min Act: -0.1700

Data Index: 4535332 (Open Web Text)

Max Activating Token Index: 295

Text #8

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.2273. Min Act: -0.1700

Data Index: 40759 (Open Web Text)

Max Activating Token Index: 509

Text #9

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.2075. Min Act: -0.1700

Data Index: 5520270 (Open Web Text)

Max Activating Token Index: 89

Text #10

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.2127. Min Act: -0.1700

Data Index: 4224604 (Open Web Text)

Max Activating Token Index: 9

Text #11

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.1587. Min Act: -0.1700

Data Index: 2807083 (Open Web Text)

Max Activating Token Index: 181

Text #12

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.1551. Min Act: -0.1700

Data Index: 335164 (Open Web Text)

Max Activating Token Index: 526

Text #13

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.2057. Min Act: -0.1700

Data Index: 6496860 (Open Web Text)

Max Activating Token Index: 433

Text #14

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.1472. Min Act: -0.1700

Data Index: 7288562 (Open Web Text)

Max Activating Token Index: 887

Text #15

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.1297. Min Act: -0.1700

Data Index: 748949 (Open Web Text)

Max Activating Token Index: 329

Text #16

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.1345. Min Act: -0.1700

Data Index: 6630598 (Open Web Text)

Max Activating Token Index: 406

Text #17

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.0952. Min Act: -0.1700

Data Index: 2968509 (Open Web Text)

Max Activating Token Index: 425

Text #18

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.0899. Min Act: -0.1700

Data Index: 3859211 (Open Web Text)

Max Activating Token Index: 844

Text #19

Max Range: 3.5753. Min Range: -3.5753

Max Act: 3.0935. Min Act: -0.1700

Data Index: 1799711 (Open Web Text)

Max Activating Token Index: 787