Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1126 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.6821. Min Act: -0.1700

Data Index: 6809560 (Open Web Text)

Max Activating Token Index: 554

Text #1

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.6313. Min Act: -0.1700

Data Index: 71819 (Open Web Text)

Max Activating Token Index: 262

Text #2

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.6170. Min Act: -0.1700

Data Index: 8169770 (Open Web Text)

Max Activating Token Index: 112

Text #3

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.5830. Min Act: -0.1700

Data Index: 5577405 (Open Web Text)

Max Activating Token Index: 483

Text #4

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.5370. Min Act: -0.1700

Data Index: 3206190 (Open Web Text)

Max Activating Token Index: 979

Text #5

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.5314. Min Act: -0.1700

Data Index: 4281686 (Open Web Text)

Max Activating Token Index: 460

Text #6

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.5409. Min Act: -0.1700

Data Index: 3010841 (Open Web Text)

Max Activating Token Index: 478

Text #7

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.5176. Min Act: -0.1700

Data Index: 7026413 (Open Web Text)

Max Activating Token Index: 433

Text #8

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.5241. Min Act: -0.1700

Data Index: 7215229 (Open Web Text)

Max Activating Token Index: 585

Text #9

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.5156. Min Act: -0.1700

Data Index: 777895 (Open Web Text)

Max Activating Token Index: 942

Text #10

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4821. Min Act: -0.1700

Data Index: 6037927 (Open Web Text)

Max Activating Token Index: 335

Text #11

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4723. Min Act: -0.1700

Data Index: 7308821 (Open Web Text)

Max Activating Token Index: 515

Text #12

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4648. Min Act: -0.1700

Data Index: 6019438 (Open Web Text)

Max Activating Token Index: 334

Text #13

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4490. Min Act: -0.1700

Data Index: 1846473 (Open Web Text)

Max Activating Token Index: 866

Text #14

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4634. Min Act: -0.1700

Data Index: 7008628 (Open Web Text)

Max Activating Token Index: 237

Text #15

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4535. Min Act: -0.1700

Data Index: 5552678 (Open Web Text)

Max Activating Token Index: 652

Text #16

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4154. Min Act: -0.1700

Data Index: 1715223 (Open Web Text)

Max Activating Token Index: 883

Text #17

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4468. Min Act: -0.1700

Data Index: 2285901 (Open Web Text)

Max Activating Token Index: 352

Text #18

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4410. Min Act: -0.1700

Data Index: 6497015 (Open Web Text)

Max Activating Token Index: 348

Text #19

Max Range: 3.6821. Min Range: -3.6821

Max Act: 3.4190. Min Act: -0.1700

Data Index: 1800867 (Open Web Text)

Max Activating Token Index: 876