Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 391 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.6237. Min Act: -0.1700

Data Index: 8635463 (Open Web Text)

Max Activating Token Index: 303

Text #1

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.2398. Min Act: -0.1700

Data Index: 3921879 (Open Web Text)

Max Activating Token Index: 522

Text #2

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.2102. Min Act: -0.1700

Data Index: 6168937 (Open Web Text)

Max Activating Token Index: 526

Text #3

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.1667. Min Act: -0.1700

Data Index: 1998268 (Open Web Text)

Max Activating Token Index: 683

Text #4

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.0601. Min Act: -0.1700

Data Index: 6734964 (Open Web Text)

Max Activating Token Index: 922

Text #5

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.1491. Min Act: -0.1700

Data Index: 7171659 (Open Web Text)

Max Activating Token Index: 721

Text #6

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.1168. Min Act: -0.1700

Data Index: 592347 (Open Web Text)

Max Activating Token Index: 644

Text #7

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.0731. Min Act: -0.1700

Data Index: 7564646 (Open Web Text)

Max Activating Token Index: 370

Text #8

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.0476. Min Act: -0.1700

Data Index: 5928111 (Open Web Text)

Max Activating Token Index: 387

Text #9

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.0079. Min Act: -0.1700

Data Index: 4633850 (Open Web Text)

Max Activating Token Index: 486

Text #10

Max Range: 4.6237. Min Range: -4.6237

Max Act: 4.0111. Min Act: -0.1700

Data Index: 6186314 (Open Web Text)

Max Activating Token Index: 887

Text #11

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.9460. Min Act: -0.1700

Data Index: 1652877 (Open Web Text)

Max Activating Token Index: 700

Text #12

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.8614. Min Act: -0.1700

Data Index: 6639487 (Open Web Text)

Max Activating Token Index: 942

Text #13

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.9404. Min Act: -0.1700

Data Index: 3774442 (Open Web Text)

Max Activating Token Index: 772

Text #14

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.8539. Min Act: -0.1700

Data Index: 500276 (Open Web Text)

Max Activating Token Index: 465

Text #15

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.8806. Min Act: -0.1700

Data Index: 2630167 (Open Web Text)

Max Activating Token Index: 895

Text #16

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.8086. Min Act: -0.1700

Data Index: 3289235 (Open Web Text)

Max Activating Token Index: 900

Text #17

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.8020. Min Act: -0.1700

Data Index: 4884703 (Open Web Text)

Max Activating Token Index: 926

Text #18

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.7306. Min Act: -0.1700

Data Index: 1198409 (Open Web Text)

Max Activating Token Index: 895

Text #19

Max Range: 4.6237. Min Range: -4.6237

Max Act: 3.8079. Min Act: -0.1700

Data Index: 1130322 (Open Web Text)

Max Activating Token Index: 167