Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 652 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.5319. Min Act: -0.1700

Data Index: 3041087 (Open Web Text)

Max Activating Token Index: 775

Text #1

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.4662. Min Act: -0.1700

Data Index: 2553207 (Open Web Text)

Max Activating Token Index: 235

Text #2

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3883. Min Act: -0.1700

Data Index: 6806821 (Open Web Text)

Max Activating Token Index: 577

Text #3

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3742. Min Act: -0.1700

Data Index: 6932522 (Open Web Text)

Max Activating Token Index: 45

Text #4

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3676. Min Act: -0.1700

Data Index: 5075559 (Open Web Text)

Max Activating Token Index: 407

Text #5

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3771. Min Act: -0.1700

Data Index: 993294 (Open Web Text)

Max Activating Token Index: 176

Text #6

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3653. Min Act: -0.1700

Data Index: 6243656 (Open Web Text)

Max Activating Token Index: 944

Text #7

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3310. Min Act: -0.1700

Data Index: 6606004 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3270. Min Act: -0.1700

Data Index: 8686043 (Open Web Text)

Max Activating Token Index: 209

Text #9

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3838. Min Act: -0.1700

Data Index: 7778397 (Open Web Text)

Max Activating Token Index: 1017

Text #10

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3204. Min Act: -0.1700

Data Index: 6941686 (Open Web Text)

Max Activating Token Index: 909

Text #11

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3241. Min Act: -0.1700

Data Index: 5855984 (Open Web Text)

Max Activating Token Index: 744

Text #12

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3237. Min Act: -0.1700

Data Index: 3694118 (Open Web Text)

Max Activating Token Index: 532

Text #13

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3191. Min Act: -0.1700

Data Index: 2535886 (Open Web Text)

Max Activating Token Index: 629

Text #14

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2964. Min Act: -0.1700

Data Index: 3093668 (Open Web Text)

Max Activating Token Index: 925

Text #15

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2944. Min Act: -0.1700

Data Index: 5761893 (Open Web Text)

Max Activating Token Index: 932

Text #16

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3313. Min Act: -0.1700

Data Index: 8510135 (Open Web Text)

Max Activating Token Index: 665

Text #17

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3161. Min Act: -0.1700

Data Index: 6437843 (Open Web Text)

Max Activating Token Index: 665

Text #18

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2873. Min Act: -0.1700

Data Index: 5314202 (Open Web Text)

Max Activating Token Index: 404

Text #19

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3128. Min Act: -0.1700

Data Index: 7608289 (Open Web Text)

Max Activating Token Index: 57