Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2866 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0103. Min Range: -4.0103

Max Act: 4.0103. Min Act: -0.1700

Data Index: 675060 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9691. Min Act: -0.1700

Data Index: 8214450 (Open Web Text)

Max Activating Token Index: 455

Text #2

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9799. Min Act: -0.1700

Data Index: 3300774 (Open Web Text)

Max Activating Token Index: 34

Text #3

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9492. Min Act: -0.1700

Data Index: 7128169 (Open Web Text)

Max Activating Token Index: 689

Text #4

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9471. Min Act: -0.1700

Data Index: 693431 (Open Web Text)

Max Activating Token Index: 38

Text #5

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9712. Min Act: -0.1700

Data Index: 7864845 (Open Web Text)

Max Activating Token Index: 213

Text #6

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9650. Min Act: -0.1700

Data Index: 6520821 (Open Web Text)

Max Activating Token Index: 329

Text #7

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9495. Min Act: -0.1700

Data Index: 908858 (Open Web Text)

Max Activating Token Index: 132

Text #8

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9589. Min Act: -0.1700

Data Index: 2724448 (Open Web Text)

Max Activating Token Index: 674

Text #9

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9538. Min Act: -0.1700

Data Index: 2482038 (Open Web Text)

Max Activating Token Index: 180

Text #10

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9402. Min Act: -0.1700

Data Index: 6335559 (Open Web Text)

Max Activating Token Index: 26

Text #11

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9064. Min Act: -0.1700

Data Index: 4936827 (Open Web Text)

Max Activating Token Index: 913

Text #12

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9210. Min Act: -0.1700

Data Index: 4841366 (Open Web Text)

Max Activating Token Index: 828

Text #13

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9063. Min Act: -0.1700

Data Index: 4599091 (Open Web Text)

Max Activating Token Index: 396

Text #14

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9232. Min Act: -0.1700

Data Index: 4390724 (Open Web Text)

Max Activating Token Index: 425

Text #15

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9277. Min Act: -0.1700

Data Index: 3971791 (Open Web Text)

Max Activating Token Index: 54

Text #16

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9341. Min Act: -0.1700

Data Index: 3590619 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.8943. Min Act: -0.1700

Data Index: 2341280 (Open Web Text)

Max Activating Token Index: 226

Text #18

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9191. Min Act: -0.1700

Data Index: 114972 (Open Web Text)

Max Activating Token Index: 429

Text #19

Max Range: 4.0103. Min Range: -4.0103

Max Act: 3.9161. Min Act: -0.1700

Data Index: 3263704 (Open Web Text)

Max Activating Token Index: 592