Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 198 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.5965. Min Act: -0.1700

Data Index: 1950321 (Open Web Text)

Max Activating Token Index: 308

Text #1

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.5619. Min Act: -0.1700

Data Index: 660310 (Open Web Text)

Max Activating Token Index: 938

Text #2

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.5535. Min Act: -0.1700

Data Index: 7426305 (Open Web Text)

Max Activating Token Index: 768

Text #3

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.5255. Min Act: -0.1700

Data Index: 5546969 (Open Web Text)

Max Activating Token Index: 614

Text #4

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.4923. Min Act: -0.1700

Data Index: 8607158 (Open Web Text)

Max Activating Token Index: 135

Text #5

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.4516. Min Act: -0.1700

Data Index: 2076154 (Open Web Text)

Max Activating Token Index: 429

Text #6

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.4404. Min Act: -0.1700

Data Index: 963927 (Open Web Text)

Max Activating Token Index: 111

Text #7

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3675. Min Act: -0.1700

Data Index: 6155679 (Open Web Text)

Max Activating Token Index: 632

Text #8

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.4383. Min Act: -0.1700

Data Index: 4801725 (Open Web Text)

Max Activating Token Index: 884

Text #9

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3858. Min Act: -0.1700

Data Index: 2569420 (Open Web Text)

Max Activating Token Index: 476

Text #10

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.4094. Min Act: -0.1700

Data Index: 2438175 (Open Web Text)

Max Activating Token Index: 268

Text #11

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3760. Min Act: -0.1700

Data Index: 1805802 (Open Web Text)

Max Activating Token Index: 921

Text #12

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.4526. Min Act: -0.1700

Data Index: 3498745 (Open Web Text)

Max Activating Token Index: 727

Text #13

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3525. Min Act: -0.1700

Data Index: 5862172 (Open Web Text)

Max Activating Token Index: 844

Text #14

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3641. Min Act: -0.1700

Data Index: 6366559 (Open Web Text)

Max Activating Token Index: 563

Text #15

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3662. Min Act: -0.1700

Data Index: 1602061 (Open Web Text)

Max Activating Token Index: 846

Text #16

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3431. Min Act: -0.1700

Data Index: 1494485 (Open Web Text)

Max Activating Token Index: 431

Text #17

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3298. Min Act: -0.1700

Data Index: 540080 (Open Web Text)

Max Activating Token Index: 575

Text #18

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3518. Min Act: -0.1700

Data Index: 832152 (Open Web Text)

Max Activating Token Index: 734

Text #19

Max Range: 3.5965. Min Range: -3.5965

Max Act: 3.3097. Min Act: -0.1700

Data Index: 5510291 (Open Web Text)

Max Activating Token Index: 616