Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3934 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.8223. Min Act: -0.0000

Data Index: 1123249 (The Pile)

Max Activating Token Index: 689

Text #1

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.4467. Min Act: -0.0000

Data Index: 1690637 (The Pile)

Max Activating Token Index: 14

Text #2

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.4151. Min Act: -0.0000

Data Index: 1972562 (The Pile)

Max Activating Token Index: 369

Text #3

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.4204. Min Act: -0.0000

Data Index: 878127 (The Pile)

Max Activating Token Index: 452

Text #4

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.4189. Min Act: -0.0000

Data Index: 159107 (The Pile)

Max Activating Token Index: 388

Text #5

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3926. Min Act: -0.0000

Data Index: 613158 (The Pile)

Max Activating Token Index: 240

Text #6

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3973. Min Act: -0.0000

Data Index: 914677 (The Pile)

Max Activating Token Index: 433

Text #7

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3791. Min Act: -0.0000

Data Index: 1323033 (The Pile)

Max Activating Token Index: 580

Text #8

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3875. Min Act: -0.0000

Data Index: 1021400 (The Pile)

Max Activating Token Index: 372

Text #9

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3951. Min Act: -0.0000

Data Index: 1548928 (The Pile)

Max Activating Token Index: 439

Text #10

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3486. Min Act: -0.0000

Data Index: 552697 (The Pile)

Max Activating Token Index: 224

Text #11

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3446. Min Act: -0.0000

Data Index: 658084 (The Pile)

Max Activating Token Index: 487

Text #12

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3254. Min Act: -0.0000

Data Index: 259010 (The Pile)

Max Activating Token Index: 747

Text #13

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3448. Min Act: -0.0000

Data Index: 862566 (The Pile)

Max Activating Token Index: 16

Text #14

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3604. Min Act: -0.0000

Data Index: 272086 (The Pile)

Max Activating Token Index: 419

Text #15

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3229. Min Act: -0.0000

Data Index: 330458 (The Pile)

Max Activating Token Index: 12

Text #16

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3265. Min Act: -0.0000

Data Index: 1781578 (The Pile)

Max Activating Token Index: 328

Text #17

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3238. Min Act: -0.0000

Data Index: 1367436 (The Pile)

Max Activating Token Index: 59

Text #18

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3292. Min Act: -0.0000

Data Index: 919045 (The Pile)

Max Activating Token Index: 351

Text #19

Max Range: 0.8223. Min Range: -0.8223

Max Act: 0.3463. Min Act: 0.0000

Data Index: 1261748 (The Pile)

Max Activating Token Index: 12