Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4846 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.7110. Min Act: -0.0000

Data Index: 800664 (The Pile)

Max Activating Token Index: 421

Text #1

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.6320. Min Act: -0.0000

Data Index: 433153 (The Pile)

Max Activating Token Index: 338

Text #2

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.6011. Min Act: -0.0000

Data Index: 1463937 (The Pile)

Max Activating Token Index: 910

Text #3

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.6337. Min Act: -0.0000

Data Index: 61971 (The Pile)

Max Activating Token Index: 676

Text #4

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.6286. Min Act: -0.0000

Data Index: 569805 (The Pile)

Max Activating Token Index: 957

Text #5

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5906. Min Act: -0.0000

Data Index: 876850 (The Pile)

Max Activating Token Index: 779

Text #6

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5814. Min Act: -0.0000

Data Index: 1980916 (The Pile)

Max Activating Token Index: 332

Text #7

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.6006. Min Act: -0.0000

Data Index: 1436336 (The Pile)

Max Activating Token Index: 878

Text #8

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5756. Min Act: -0.0000

Data Index: 1589189 (The Pile)

Max Activating Token Index: 305

Text #9

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5715. Min Act: -0.0000

Data Index: 1342917 (The Pile)

Max Activating Token Index: 550

Text #10

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5714. Min Act: -0.0000

Data Index: 230841 (The Pile)

Max Activating Token Index: 93

Text #11

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5516. Min Act: -0.0000

Data Index: 564698 (The Pile)

Max Activating Token Index: 421

Text #12

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5696. Min Act: -0.0000

Data Index: 806922 (The Pile)

Max Activating Token Index: 811

Text #13

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5442. Min Act: -0.0000

Data Index: 1208484 (The Pile)

Max Activating Token Index: 483

Text #14

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5367. Min Act: -0.0000

Data Index: 1427091 (The Pile)

Max Activating Token Index: 858

Text #15

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5445. Min Act: -0.0000

Data Index: 281663 (The Pile)

Max Activating Token Index: 256

Text #16

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5365. Min Act: -0.0000

Data Index: 1299927 (The Pile)

Max Activating Token Index: 834

Text #17

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5212. Min Act: -0.0000

Data Index: 1381232 (The Pile)

Max Activating Token Index: 417

Text #18

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5145. Min Act: -0.0000

Data Index: 625410 (The Pile)

Max Activating Token Index: 431

Text #19

Max Range: 0.7110. Min Range: -0.7110

Max Act: 0.5297. Min Act: -0.0000

Data Index: 1897158 (The Pile)

Max Activating Token Index: 997