Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1796 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.6753. Min Act: -0.1700

Data Index: 5330078 (Open Web Text)

Max Activating Token Index: 495

Text #1

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.5640. Min Act: -0.1700

Data Index: 1565633 (Open Web Text)

Max Activating Token Index: 461

Text #2

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.5487. Min Act: -0.1700

Data Index: 1283913 (Open Web Text)

Max Activating Token Index: 471

Text #3

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.4826. Min Act: -0.1700

Data Index: 2542289 (Open Web Text)

Max Activating Token Index: 602

Text #4

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.4854. Min Act: -0.1700

Data Index: 8292635 (Open Web Text)

Max Activating Token Index: 224

Text #5

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.4423. Min Act: -0.1700

Data Index: 5463295 (Open Web Text)

Max Activating Token Index: 418

Text #6

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.4583. Min Act: -0.1700

Data Index: 2653464 (Open Web Text)

Max Activating Token Index: 602

Text #7

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.4025. Min Act: -0.1700

Data Index: 1222945 (Open Web Text)

Max Activating Token Index: 842

Text #8

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3660. Min Act: -0.1700

Data Index: 4503155 (Open Web Text)

Max Activating Token Index: 985

Text #9

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3853. Min Act: -0.1700

Data Index: 4650137 (Open Web Text)

Max Activating Token Index: 284

Text #10

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3561. Min Act: -0.1700

Data Index: 5160702 (Open Web Text)

Max Activating Token Index: 157

Text #11

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3837. Min Act: -0.1700

Data Index: 7719123 (Open Web Text)

Max Activating Token Index: 580

Text #12

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3755. Min Act: -0.1700

Data Index: 3627079 (Open Web Text)

Max Activating Token Index: 133

Text #13

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3230. Min Act: -0.1700

Data Index: 2405047 (Open Web Text)

Max Activating Token Index: 915

Text #14

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3195. Min Act: -0.1700

Data Index: 4179490 (Open Web Text)

Max Activating Token Index: 857

Text #15

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.2825. Min Act: -0.1700

Data Index: 3525006 (Open Web Text)

Max Activating Token Index: 912

Text #16

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3235. Min Act: -0.1700

Data Index: 3496953 (Open Web Text)

Max Activating Token Index: 218

Text #17

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3222. Min Act: -0.1700

Data Index: 290214 (Open Web Text)

Max Activating Token Index: 461

Text #18

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.2798. Min Act: -0.1700

Data Index: 6322343 (Open Web Text)

Max Activating Token Index: 364

Text #19

Max Range: 4.6753. Min Range: -4.6753

Max Act: 4.3044. Min Act: -0.1700

Data Index: 7729544 (Open Web Text)

Max Activating Token Index: 295