Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4048 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.9285. Min Act: 0.0000

Data Index: 1196204 (The Pile)

Max Activating Token Index: 423

Text #1

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.8353. Min Act: -0.0000

Data Index: 1677097 (The Pile)

Max Activating Token Index: 737

Text #2

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.9071. Min Act: -0.0000

Data Index: 440578 (The Pile)

Max Activating Token Index: 540

Text #3

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.8506. Min Act: -0.0000

Data Index: 94328 (The Pile)

Max Activating Token Index: 680

Text #4

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.6368. Min Act: -0.0000

Data Index: 1399308 (The Pile)

Max Activating Token Index: 661

Text #5

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5409. Min Act: -0.0000

Data Index: 1552660 (The Pile)

Max Activating Token Index: 323

Text #6

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5905. Min Act: -0.0000

Data Index: 157665 (The Pile)

Max Activating Token Index: 1011

Text #7

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5354. Min Act: -0.0000

Data Index: 1913770 (The Pile)

Max Activating Token Index: 880

Text #8

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5272. Min Act: -0.0000

Data Index: 104064 (The Pile)

Max Activating Token Index: 493

Text #9

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5554. Min Act: 0.0000

Data Index: 865474 (The Pile)

Max Activating Token Index: 679

Text #10

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5453. Min Act: -0.0000

Data Index: 1428241 (The Pile)

Max Activating Token Index: 489

Text #11

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5603. Min Act: -0.0000

Data Index: 274271 (The Pile)

Max Activating Token Index: 621

Text #12

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5546. Min Act: -0.0000

Data Index: 397973 (The Pile)

Max Activating Token Index: 777

Text #13

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5454. Min Act: -0.0000

Data Index: 1194919 (The Pile)

Max Activating Token Index: 496

Text #14

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.4833. Min Act: -0.0000

Data Index: 119518 (The Pile)

Max Activating Token Index: 788

Text #15

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5417. Min Act: -0.0000

Data Index: 1859309 (The Pile)

Max Activating Token Index: 451

Text #16

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5417. Min Act: -0.0000

Data Index: 824347 (The Pile)

Max Activating Token Index: 444

Text #17

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5123. Min Act: 0.0000

Data Index: 1270544 (The Pile)

Max Activating Token Index: 987

Text #18

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.5165. Min Act: -0.0000

Data Index: 1131656 (The Pile)

Max Activating Token Index: 947

Text #19

Max Range: 0.9285. Min Range: -0.9285

Max Act: 0.4814. Min Act: 0.0000

Data Index: 1142817 (The Pile)

Max Activating Token Index: 888