Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 815 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0286. Min Range: -1.0286

Max Act: 1.0286. Min Act: -0.0000

Data Index: 1933842 (The Pile)

Max Activating Token Index: 539

Text #1

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.9275. Min Act: -0.0000

Data Index: 863811 (The Pile)

Max Activating Token Index: 489

Text #2

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.8789. Min Act: -0.0000

Data Index: 1001609 (The Pile)

Max Activating Token Index: 107

Text #3

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.8925. Min Act: -0.0000

Data Index: 1807016 (The Pile)

Max Activating Token Index: 531

Text #4

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.9367. Min Act: -0.0000

Data Index: 1823937 (The Pile)

Max Activating Token Index: 956

Text #5

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.8064. Min Act: -0.0000

Data Index: 616416 (The Pile)

Max Activating Token Index: 841

Text #6

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7621. Min Act: -0.0000

Data Index: 1835409 (The Pile)

Max Activating Token Index: 63

Text #7

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7675. Min Act: -0.0000

Data Index: 1236402 (The Pile)

Max Activating Token Index: 256

Text #8

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.8132. Min Act: -0.0000

Data Index: 1312330 (The Pile)

Max Activating Token Index: 645

Text #9

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7811. Min Act: -0.0000

Data Index: 527901 (The Pile)

Max Activating Token Index: 916

Text #10

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7582. Min Act: -0.0000

Data Index: 939734 (The Pile)

Max Activating Token Index: 991

Text #11

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7311. Min Act: 0.0000

Data Index: 12495 (The Pile)

Max Activating Token Index: 237

Text #12

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7762. Min Act: -0.0000

Data Index: 31733 (The Pile)

Max Activating Token Index: 768

Text #13

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7565. Min Act: -0.0000

Data Index: 810133 (The Pile)

Max Activating Token Index: 486

Text #14

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7315. Min Act: -0.0000

Data Index: 1903795 (The Pile)

Max Activating Token Index: 225

Text #15

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7812. Min Act: -0.0000

Data Index: 723183 (The Pile)

Max Activating Token Index: 520

Text #16

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7569. Min Act: -0.0000

Data Index: 308438 (The Pile)

Max Activating Token Index: 423

Text #17

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7062. Min Act: -0.0000

Data Index: 1184099 (The Pile)

Max Activating Token Index: 288

Text #18

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7476. Min Act: -0.0000

Data Index: 1598454 (The Pile)

Max Activating Token Index: 390

Text #19

Max Range: 1.0286. Min Range: -1.0286

Max Act: 0.7162. Min Act: -0.0000

Data Index: 295950 (The Pile)

Max Activating Token Index: 183