Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 765 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.3911. Min Act: -0.1700

Data Index: 5784783 (Open Web Text)

Max Activating Token Index: 273

Text #1

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.3157. Min Act: -0.1700

Data Index: 2708223 (Open Web Text)

Max Activating Token Index: 745

Text #2

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.3526. Min Act: -0.1700

Data Index: 6243914 (Open Web Text)

Max Activating Token Index: 136

Text #3

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.3228. Min Act: -0.1700

Data Index: 8120014 (Open Web Text)

Max Activating Token Index: 815

Text #4

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.3165. Min Act: -0.1700

Data Index: 2886694 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.3264. Min Act: -0.1700

Data Index: 7385648 (Open Web Text)

Max Activating Token Index: 287

Text #6

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2928. Min Act: -0.1700

Data Index: 8383617 (Open Web Text)

Max Activating Token Index: 234

Text #7

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.3176. Min Act: -0.1700

Data Index: 2384331 (Open Web Text)

Max Activating Token Index: 684

Text #8

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2792. Min Act: -0.1700

Data Index: 5556942 (Open Web Text)

Max Activating Token Index: 728

Text #9

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2604. Min Act: -0.1700

Data Index: 7189144 (Open Web Text)

Max Activating Token Index: 494

Text #10

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2481. Min Act: -0.1700

Data Index: 8611252 (Open Web Text)

Max Activating Token Index: 444

Text #11

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2578. Min Act: -0.1700

Data Index: 4328071 (Open Web Text)

Max Activating Token Index: 360

Text #12

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2712. Min Act: -0.1700

Data Index: 57363 (Open Web Text)

Max Activating Token Index: 291

Text #13

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2490. Min Act: -0.1700

Data Index: 6597317 (Open Web Text)

Max Activating Token Index: 139

Text #14

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2350. Min Act: -0.1700

Data Index: 6247314 (Open Web Text)

Max Activating Token Index: 904

Text #15

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2473. Min Act: -0.1700

Data Index: 8146173 (Open Web Text)

Max Activating Token Index: 278

Text #16

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2082. Min Act: -0.1700

Data Index: 7778496 (Open Web Text)

Max Activating Token Index: 516

Text #17

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2268. Min Act: -0.1700

Data Index: 7869224 (Open Web Text)

Max Activating Token Index: 661

Text #18

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.1953. Min Act: -0.1700

Data Index: 5751927 (Open Web Text)

Max Activating Token Index: 1009

Text #19

Max Range: 4.3911. Min Range: -4.3911

Max Act: 4.2067. Min Act: -0.1700

Data Index: 3336754 (Open Web Text)

Max Activating Token Index: 370