Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1867 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.9255. Min Act: -0.0000

Data Index: 531055 (The Pile)

Max Activating Token Index: 819

Text #1

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.7147. Min Act: -0.0000

Data Index: 1156883 (The Pile)

Max Activating Token Index: 696

Text #2

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.5717. Min Act: -0.0000

Data Index: 109322 (The Pile)

Max Activating Token Index: 445

Text #3

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.5102. Min Act: -0.0000

Data Index: 661202 (The Pile)

Max Activating Token Index: 385

Text #4

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.5100. Min Act: -0.0000

Data Index: 346213 (The Pile)

Max Activating Token Index: 1000

Text #5

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4532. Min Act: -0.0000

Data Index: 1561594 (The Pile)

Max Activating Token Index: 993

Text #6

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.1597. Min Act: -0.0000

Data Index: 1248949 (The Pile)

Max Activating Token Index: 716

Text #7

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4019. Min Act: -0.0000

Data Index: 1845746 (The Pile)

Max Activating Token Index: 529

Text #8

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.3692. Min Act: -0.0000

Data Index: 1458453 (The Pile)

Max Activating Token Index: 619

Text #9

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4330. Min Act: -0.0000

Data Index: 1435407 (The Pile)

Max Activating Token Index: 273

Text #10

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.3928. Min Act: -0.0000

Data Index: 117403 (The Pile)

Max Activating Token Index: 749

Text #11

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4253. Min Act: -0.0000

Data Index: 1061434 (The Pile)

Max Activating Token Index: 143

Text #12

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.3097. Min Act: -0.0000

Data Index: 1201633 (The Pile)

Max Activating Token Index: 943

Text #13

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.1989. Min Act: -0.0000

Data Index: 1467010 (The Pile)

Max Activating Token Index: 442

Text #14

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4615. Min Act: -0.0000

Data Index: 208642 (The Pile)

Max Activating Token Index: 632

Text #15

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4227. Min Act: -0.0000

Data Index: 774258 (The Pile)

Max Activating Token Index: 290

Text #16

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4174. Min Act: -0.0000

Data Index: 1169204 (The Pile)

Max Activating Token Index: 386

Text #17

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4293. Min Act: -0.0000

Data Index: 1876655 (The Pile)

Max Activating Token Index: 688

Text #18

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.2745. Min Act: -0.0000

Data Index: 694943 (The Pile)

Max Activating Token Index: 809

Text #19

Max Range: 1.9255. Min Range: -1.9255

Max Act: 1.4321. Min Act: -0.0000

Data Index: 105804 (The Pile)

Max Activating Token Index: 488