Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3736 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.9689. Min Act: -0.0000

Data Index: 1934416 (The Pile)

Max Activating Token Index: 576

Text #1

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.9399. Min Act: -0.0000

Data Index: 646032 (The Pile)

Max Activating Token Index: 906

Text #2

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7539. Min Act: -0.0000

Data Index: 1315039 (The Pile)

Max Activating Token Index: 170

Text #3

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.9146. Min Act: -0.0000

Data Index: 1712371 (The Pile)

Max Activating Token Index: 1005

Text #4

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.8161. Min Act: -0.0000

Data Index: 1277830 (The Pile)

Max Activating Token Index: 349

Text #5

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7147. Min Act: -0.0000

Data Index: 1488565 (The Pile)

Max Activating Token Index: 47

Text #6

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.8112. Min Act: -0.0000

Data Index: 1484956 (The Pile)

Max Activating Token Index: 711

Text #7

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.8224. Min Act: -0.0000

Data Index: 1927942 (The Pile)

Max Activating Token Index: 318

Text #8

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7430. Min Act: -0.0000

Data Index: 454361 (The Pile)

Max Activating Token Index: 786

Text #9

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.8195. Min Act: -0.0000

Data Index: 1955867 (The Pile)

Max Activating Token Index: 189

Text #10

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7964. Min Act: -0.0000

Data Index: 1609871 (The Pile)

Max Activating Token Index: 994

Text #11

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7875. Min Act: -0.0000

Data Index: 405180 (The Pile)

Max Activating Token Index: 390

Text #12

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.6784. Min Act: -0.0000

Data Index: 1357839 (The Pile)

Max Activating Token Index: 453

Text #13

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.5768. Min Act: -0.0000

Data Index: 1086245 (The Pile)

Max Activating Token Index: 687

Text #14

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.6666. Min Act: -0.0000

Data Index: 68915 (The Pile)

Max Activating Token Index: 1003

Text #15

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.6609. Min Act: -0.0000

Data Index: 1909352 (The Pile)

Max Activating Token Index: 162

Text #16

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7558. Min Act: -0.0000

Data Index: 1850546 (The Pile)

Max Activating Token Index: 300

Text #17

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7557. Min Act: -0.0000

Data Index: 105715 (The Pile)

Max Activating Token Index: 1022

Text #18

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7567. Min Act: -0.0000

Data Index: 1402454 (The Pile)

Max Activating Token Index: 466

Text #19

Max Range: 1.9689. Min Range: -1.9689

Max Act: 1.7485. Min Act: -0.0000

Data Index: 67173 (The Pile)

Max Activating Token Index: 255