Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 192 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.1269. Min Act: -0.0000

Data Index: 1374144 (The Pile)

Max Activating Token Index: 763

Text #1

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.2293. Min Act: -0.0000

Data Index: 1554492 (The Pile)

Max Activating Token Index: 634

Text #2

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.1861. Min Act: -0.0000

Data Index: 404590 (The Pile)

Max Activating Token Index: 153

Text #3

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.1147. Min Act: -0.0000

Data Index: 1659975 (The Pile)

Max Activating Token Index: 590

Text #4

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0497. Min Act: -0.0000

Data Index: 231194 (The Pile)

Max Activating Token Index: 541

Text #5

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0472. Min Act: -0.0000

Data Index: 341302 (The Pile)

Max Activating Token Index: 338

Text #6

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0986. Min Act: -0.0000

Data Index: 1762944 (The Pile)

Max Activating Token Index: 221

Text #7

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0397. Min Act: -0.0000

Data Index: 570004 (The Pile)

Max Activating Token Index: 683

Text #8

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0823. Min Act: -0.0000

Data Index: 676823 (The Pile)

Max Activating Token Index: 865

Text #9

Max Range: 1.2293. Min Range: -1.2293

Max Act: 0.9980. Min Act: -0.0000

Data Index: 454620 (The Pile)

Max Activating Token Index: 854

Text #10

Max Range: 1.2293. Min Range: -1.2293

Max Act: 0.9644. Min Act: -0.0000

Data Index: 861332 (The Pile)

Max Activating Token Index: 505

Text #11

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0426. Min Act: -0.0000

Data Index: 383942 (The Pile)

Max Activating Token Index: 127

Text #12

Max Range: 1.2293. Min Range: -1.2293

Max Act: 0.9537. Min Act: -0.0000

Data Index: 1870746 (The Pile)

Max Activating Token Index: 549

Text #13

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0168. Min Act: -0.0000

Data Index: 898031 (The Pile)

Max Activating Token Index: 574

Text #14

Max Range: 1.2293. Min Range: -1.2293

Max Act: 0.9357. Min Act: -0.0000

Data Index: 799111 (The Pile)

Max Activating Token Index: 605

Text #15

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0094. Min Act: -0.0000

Data Index: 1029316 (The Pile)

Max Activating Token Index: 373

Text #16

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0219. Min Act: -0.0000

Data Index: 1615619 (The Pile)

Max Activating Token Index: 754

Text #17

Max Range: 1.2293. Min Range: -1.2293

Max Act: 0.9777. Min Act: -0.0000

Data Index: 1533443 (The Pile)

Max Activating Token Index: 446

Text #18

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0746. Min Act: -0.0000

Data Index: 391081 (The Pile)

Max Activating Token Index: 1016

Text #19

Max Range: 1.2293. Min Range: -1.2293

Max Act: 1.0157. Min Act: -0.0000

Data Index: 789188 (The Pile)

Max Activating Token Index: 643