Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 351 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.8271. Min Act: -0.0000

Data Index: 1063153 (The Pile)

Max Activating Token Index: 975

Text #1

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6616. Min Act: -0.0000

Data Index: 446824 (The Pile)

Max Activating Token Index: 453

Text #2

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6362. Min Act: -0.0000

Data Index: 1698169 (The Pile)

Max Activating Token Index: 263

Text #3

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.7005. Min Act: -0.0000

Data Index: 602326 (The Pile)

Max Activating Token Index: 765

Text #4

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6428. Min Act: -0.0000

Data Index: 954997 (The Pile)

Max Activating Token Index: 339

Text #5

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6508. Min Act: -0.0000

Data Index: 269098 (The Pile)

Max Activating Token Index: 467

Text #6

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6796. Min Act: -0.0000

Data Index: 1806550 (The Pile)

Max Activating Token Index: 230

Text #7

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6321. Min Act: -0.0000

Data Index: 1834947 (The Pile)

Max Activating Token Index: 249

Text #8

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6309. Min Act: -0.0000

Data Index: 424897 (The Pile)

Max Activating Token Index: 356

Text #9

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6339. Min Act: -0.0000

Data Index: 551714 (The Pile)

Max Activating Token Index: 111

Text #10

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6637. Min Act: -0.0000

Data Index: 421934 (The Pile)

Max Activating Token Index: 174

Text #11

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.5916. Min Act: -0.0000

Data Index: 739455 (The Pile)

Max Activating Token Index: 489

Text #12

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6119. Min Act: -0.0000

Data Index: 1865703 (The Pile)

Max Activating Token Index: 290

Text #13

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.5662. Min Act: -0.0000

Data Index: 1445602 (The Pile)

Max Activating Token Index: 330

Text #14

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.5643. Min Act: -0.0000

Data Index: 1105280 (The Pile)

Max Activating Token Index: 99

Text #15

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.5567. Min Act: -0.0000

Data Index: 580467 (The Pile)

Max Activating Token Index: 433

Text #16

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.6153. Min Act: -0.0000

Data Index: 664211 (The Pile)

Max Activating Token Index: 231

Text #17

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.5820. Min Act: -0.0000

Data Index: 1131499 (The Pile)

Max Activating Token Index: 105

Text #18

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.5233. Min Act: -0.0000

Data Index: 907067 (The Pile)

Max Activating Token Index: 998

Text #19

Max Range: 0.8271. Min Range: -0.8271

Max Act: 0.5613. Min Act: -0.0000

Data Index: 1065467 (The Pile)

Max Activating Token Index: 51