Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4458 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0418. Min Range: -1.0418

Max Act: 1.0418. Min Act: -0.0000

Data Index: 1280354 (The Pile)

Max Activating Token Index: 738

Text #1

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.9571. Min Act: -0.0000

Data Index: 1842448 (The Pile)

Max Activating Token Index: 200

Text #2

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8702. Min Act: -0.0000

Data Index: 1030503 (The Pile)

Max Activating Token Index: 793

Text #3

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.9519. Min Act: -0.0000

Data Index: 1452603 (The Pile)

Max Activating Token Index: 12

Text #4

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.9021. Min Act: -0.0000

Data Index: 838132 (The Pile)

Max Activating Token Index: 571

Text #5

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8797. Min Act: -0.0000

Data Index: 465196 (The Pile)

Max Activating Token Index: 920

Text #6

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8723. Min Act: -0.0000

Data Index: 180857 (The Pile)

Max Activating Token Index: 214

Text #7

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8338. Min Act: -0.0000

Data Index: 1547729 (The Pile)

Max Activating Token Index: 576

Text #8

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7784. Min Act: -0.0000

Data Index: 466051 (The Pile)

Max Activating Token Index: 47

Text #9

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8654. Min Act: -0.0000

Data Index: 101517 (The Pile)

Max Activating Token Index: 731

Text #10

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8665. Min Act: -0.0000

Data Index: 1900934 (The Pile)

Max Activating Token Index: 1001

Text #11

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7411. Min Act: -0.0000

Data Index: 778521 (The Pile)

Max Activating Token Index: 912

Text #12

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8456. Min Act: -0.0000

Data Index: 129189 (The Pile)

Max Activating Token Index: 229

Text #13

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7926. Min Act: -0.0000

Data Index: 1744511 (The Pile)

Max Activating Token Index: 1021

Text #14

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.8322. Min Act: -0.0000

Data Index: 1382955 (The Pile)

Max Activating Token Index: 629

Text #15

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7024. Min Act: -0.0000

Data Index: 724330 (The Pile)

Max Activating Token Index: 208

Text #16

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7421. Min Act: -0.0000

Data Index: 713134 (The Pile)

Max Activating Token Index: 1021

Text #17

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7092. Min Act: -0.0000

Data Index: 1267120 (The Pile)

Max Activating Token Index: 178

Text #18

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7761. Min Act: -0.0000

Data Index: 1440431 (The Pile)

Max Activating Token Index: 388

Text #19

Max Range: 1.0418. Min Range: -1.0418

Max Act: 0.7531. Min Act: -0.0000

Data Index: 1162540 (The Pile)

Max Activating Token Index: 847