Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1081 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6858. Min Act: -0.1700

Data Index: 5476721 (Open Web Text)

Max Activating Token Index: 678

Text #1

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6835. Min Act: -0.1700

Data Index: 3645274 (Open Web Text)

Max Activating Token Index: 844

Text #2

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6644. Min Act: -0.1700

Data Index: 4544281 (Open Web Text)

Max Activating Token Index: 434

Text #3

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6620. Min Act: -0.1700

Data Index: 4282255 (Open Web Text)

Max Activating Token Index: 694

Text #4

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6498. Min Act: -0.1700

Data Index: 2978621 (Open Web Text)

Max Activating Token Index: 771

Text #5

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6453. Min Act: -0.1700

Data Index: 3531042 (Open Web Text)

Max Activating Token Index: 969

Text #6

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6415. Min Act: -0.1700

Data Index: 5930188 (Open Web Text)

Max Activating Token Index: 651

Text #7

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6295. Min Act: -0.1700

Data Index: 1808829 (Open Web Text)

Max Activating Token Index: 114

Text #8

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6309. Min Act: -0.1700

Data Index: 1932579 (Open Web Text)

Max Activating Token Index: 790

Text #9

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6233. Min Act: -0.1700

Data Index: 4439355 (Open Web Text)

Max Activating Token Index: 947

Text #10

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6316. Min Act: -0.1700

Data Index: 6236731 (Open Web Text)

Max Activating Token Index: 908

Text #11

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6183. Min Act: -0.1700

Data Index: 6087160 (Open Web Text)

Max Activating Token Index: 122

Text #12

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6250. Min Act: -0.1700

Data Index: 7711250 (Open Web Text)

Max Activating Token Index: 372

Text #13

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6221. Min Act: -0.1700

Data Index: 4872500 (Open Web Text)

Max Activating Token Index: 696

Text #14

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6124. Min Act: -0.1700

Data Index: 2521698 (Open Web Text)

Max Activating Token Index: 126

Text #15

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6127. Min Act: -0.1700

Data Index: 8614887 (Open Web Text)

Max Activating Token Index: 310

Text #16

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6079. Min Act: -0.1700

Data Index: 5702757 (Open Web Text)

Max Activating Token Index: 74

Text #17

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6182. Min Act: -0.1700

Data Index: 933005 (Open Web Text)

Max Activating Token Index: 454

Text #18

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6154. Min Act: -0.1700

Data Index: 2198636 (Open Web Text)

Max Activating Token Index: 54

Text #19

Max Range: 1.6858. Min Range: -1.6858

Max Act: 1.6053. Min Act: -0.1700

Data Index: 4681608 (Open Web Text)

Max Activating Token Index: 685