Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3027 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1805. Min Range: -5.1805

Max Act: 5.1805. Min Act: -0.1700

Data Index: 1240853 (Open Web Text)

Max Activating Token Index: 882

Text #1

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.9344. Min Act: -0.1700

Data Index: 3526172 (Open Web Text)

Max Activating Token Index: 338

Text #2

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.9375. Min Act: -0.1700

Data Index: 8425380 (Open Web Text)

Max Activating Token Index: 115

Text #3

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8842. Min Act: -0.1700

Data Index: 6704384 (Open Web Text)

Max Activating Token Index: 898

Text #4

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8646. Min Act: -0.1700

Data Index: 3979634 (Open Web Text)

Max Activating Token Index: 249

Text #5

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8790. Min Act: -0.1700

Data Index: 8423398 (Open Web Text)

Max Activating Token Index: 402

Text #6

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8892. Min Act: -0.1700

Data Index: 3495223 (Open Web Text)

Max Activating Token Index: 265

Text #7

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8048. Min Act: -0.1700

Data Index: 5362379 (Open Web Text)

Max Activating Token Index: 771

Text #8

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8349. Min Act: -0.1700

Data Index: 6321641 (Open Web Text)

Max Activating Token Index: 607

Text #9

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7883. Min Act: -0.1700

Data Index: 4866715 (Open Web Text)

Max Activating Token Index: 191

Text #10

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8300. Min Act: -0.1700

Data Index: 20573 (Open Web Text)

Max Activating Token Index: 600

Text #11

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8296. Min Act: -0.1700

Data Index: 7206414 (Open Web Text)

Max Activating Token Index: 305

Text #12

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7779. Min Act: -0.1700

Data Index: 2451707 (Open Web Text)

Max Activating Token Index: 457

Text #13

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7938. Min Act: -0.1700

Data Index: 3122302 (Open Web Text)

Max Activating Token Index: 395

Text #14

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.8059. Min Act: -0.1700

Data Index: 828611 (Open Web Text)

Max Activating Token Index: 336

Text #15

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7495. Min Act: -0.1700

Data Index: 8415797 (Open Web Text)

Max Activating Token Index: 450

Text #16

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7223. Min Act: -0.1700

Data Index: 8369119 (Open Web Text)

Max Activating Token Index: 144

Text #17

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7456. Min Act: -0.1700

Data Index: 4518957 (Open Web Text)

Max Activating Token Index: 798

Text #18

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7192. Min Act: -0.1700

Data Index: 641253 (Open Web Text)

Max Activating Token Index: 1014

Text #19

Max Range: 5.1805. Min Range: -5.1805

Max Act: 4.7563. Min Act: -0.1700

Data Index: 1114344 (Open Web Text)

Max Activating Token Index: 791