Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 106 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.8352. Min Act: -0.0000

Data Index: 1921483 (The Pile)

Max Activating Token Index: 527

Text #1

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.7087. Min Act: -0.0000

Data Index: 996775 (The Pile)

Max Activating Token Index: 350

Text #2

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.6896. Min Act: -0.0000

Data Index: 850746 (The Pile)

Max Activating Token Index: 517

Text #3

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.6821. Min Act: -0.0000

Data Index: 693927 (The Pile)

Max Activating Token Index: 494

Text #4

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.6275. Min Act: -0.0000

Data Index: 1030326 (The Pile)

Max Activating Token Index: 278

Text #5

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.6208. Min Act: -0.0000

Data Index: 754771 (The Pile)

Max Activating Token Index: 882

Text #6

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5701. Min Act: -0.0000

Data Index: 1797287 (The Pile)

Max Activating Token Index: 291

Text #7

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5655. Min Act: -0.0000

Data Index: 870766 (The Pile)

Max Activating Token Index: 619

Text #8

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5970. Min Act: -0.0000

Data Index: 1875978 (The Pile)

Max Activating Token Index: 116

Text #9

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5748. Min Act: -0.0000

Data Index: 1605356 (The Pile)

Max Activating Token Index: 461

Text #10

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5482. Min Act: -0.0000

Data Index: 779334 (The Pile)

Max Activating Token Index: 54

Text #11

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5390. Min Act: -0.0000

Data Index: 1363382 (The Pile)

Max Activating Token Index: 544

Text #12

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5273. Min Act: -0.0000

Data Index: 483381 (The Pile)

Max Activating Token Index: 761

Text #13

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5235. Min Act: -0.0000

Data Index: 1413671 (The Pile)

Max Activating Token Index: 688

Text #14

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5172. Min Act: -0.0000

Data Index: 641435 (The Pile)

Max Activating Token Index: 374

Text #15

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5425. Min Act: -0.0000

Data Index: 115495 (The Pile)

Max Activating Token Index: 231

Text #16

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5487. Min Act: -0.0000

Data Index: 394903 (The Pile)

Max Activating Token Index: 9

Text #17

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.4705. Min Act: -0.0000

Data Index: 354212 (The Pile)

Max Activating Token Index: 462

Text #18

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5098. Min Act: -0.0000

Data Index: 580628 (The Pile)

Max Activating Token Index: 155

Text #19

Max Range: 0.8352. Min Range: -0.8352

Max Act: 0.5835. Min Act: -0.0000

Data Index: 241291 (The Pile)

Max Activating Token Index: 176