Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 19 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.9817. Min Act: -0.0000

Data Index: 1098662 (The Pile)

Max Activating Token Index: 672

Text #1

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7989. Min Act: -0.0000

Data Index: 1409489 (The Pile)

Max Activating Token Index: 992

Text #2

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.8334. Min Act: -0.0000

Data Index: 1067071 (The Pile)

Max Activating Token Index: 651

Text #3

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.8557. Min Act: -0.0000

Data Index: 956308 (The Pile)

Max Activating Token Index: 484

Text #4

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.8463. Min Act: -0.0000

Data Index: 383985 (The Pile)

Max Activating Token Index: 113

Text #5

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7783. Min Act: -0.0000

Data Index: 1705742 (The Pile)

Max Activating Token Index: 220

Text #6

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7428. Min Act: -0.0000

Data Index: 458187 (The Pile)

Max Activating Token Index: 235

Text #7

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7782. Min Act: -0.0000

Data Index: 421836 (The Pile)

Max Activating Token Index: 357

Text #8

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.8152. Min Act: -0.0000

Data Index: 598628 (The Pile)

Max Activating Token Index: 626

Text #9

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7644. Min Act: -0.0000

Data Index: 1795682 (The Pile)

Max Activating Token Index: 210

Text #10

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7231. Min Act: -0.0000

Data Index: 1783611 (The Pile)

Max Activating Token Index: 406

Text #11

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.6879. Min Act: -0.0000

Data Index: 557438 (The Pile)

Max Activating Token Index: 148

Text #12

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7262. Min Act: -0.0000

Data Index: 37452 (The Pile)

Max Activating Token Index: 11

Text #13

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7083. Min Act: -0.0000

Data Index: 1041441 (The Pile)

Max Activating Token Index: 441

Text #14

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.6946. Min Act: -0.0000

Data Index: 1409351 (The Pile)

Max Activating Token Index: 164

Text #15

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7144. Min Act: -0.0000

Data Index: 294577 (The Pile)

Max Activating Token Index: 231

Text #16

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.6470. Min Act: -0.0000

Data Index: 1123529 (The Pile)

Max Activating Token Index: 162

Text #17

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.6681. Min Act: -0.0000

Data Index: 445184 (The Pile)

Max Activating Token Index: 178

Text #18

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.6827. Min Act: -0.0000

Data Index: 360220 (The Pile)

Max Activating Token Index: 639

Text #19

Max Range: 0.9817. Min Range: -0.9817

Max Act: 0.7278. Min Act: -0.0000

Data Index: 1306376 (The Pile)

Max Activating Token Index: 443