Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4276 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.9765. Min Act: -0.0000

Data Index: 1053278 (The Pile)

Max Activating Token Index: 584

Text #1

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.8658. Min Act: -0.0000

Data Index: 1881938 (The Pile)

Max Activating Token Index: 1001

Text #2

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.8197. Min Act: -0.0000

Data Index: 911831 (The Pile)

Max Activating Token Index: 849

Text #3

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7966. Min Act: -0.0000

Data Index: 578434 (The Pile)

Max Activating Token Index: 550

Text #4

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.8116. Min Act: -0.0000

Data Index: 1468274 (The Pile)

Max Activating Token Index: 676

Text #5

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7969. Min Act: -0.0000

Data Index: 74740 (The Pile)

Max Activating Token Index: 178

Text #6

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7854. Min Act: -0.0000

Data Index: 1250403 (The Pile)

Max Activating Token Index: 991

Text #7

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7321. Min Act: -0.0000

Data Index: 773506 (The Pile)

Max Activating Token Index: 764

Text #8

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7695. Min Act: -0.0000

Data Index: 1680546 (The Pile)

Max Activating Token Index: 613

Text #9

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7384. Min Act: -0.0000

Data Index: 1177536 (The Pile)

Max Activating Token Index: 770

Text #10

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7387. Min Act: -0.0000

Data Index: 546801 (The Pile)

Max Activating Token Index: 103

Text #11

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7480. Min Act: -0.0000

Data Index: 1370141 (The Pile)

Max Activating Token Index: 609

Text #12

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.6886. Min Act: -0.0000

Data Index: 1837967 (The Pile)

Max Activating Token Index: 403

Text #13

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7052. Min Act: -0.0000

Data Index: 232767 (The Pile)

Max Activating Token Index: 323

Text #14

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7230. Min Act: -0.0000

Data Index: 1932195 (The Pile)

Max Activating Token Index: 452

Text #15

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7006. Min Act: -0.0000

Data Index: 249140 (The Pile)

Max Activating Token Index: 430

Text #16

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.7459. Min Act: -0.0000

Data Index: 771349 (The Pile)

Max Activating Token Index: 275

Text #17

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.6738. Min Act: -0.0000

Data Index: 1925615 (The Pile)

Max Activating Token Index: 899

Text #18

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.6397. Min Act: -0.0000

Data Index: 95887 (The Pile)

Max Activating Token Index: 751

Text #19

Max Range: 0.9765. Min Range: -0.9765

Max Act: 0.6854. Min Act: -0.0000

Data Index: 1905515 (The Pile)

Max Activating Token Index: 81