Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2411 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.9508. Min Act: -0.0000

Data Index: 1897884 (The Pile)

Max Activating Token Index: 1022

Text #1

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.7544. Min Act: -0.0000

Data Index: 893305 (The Pile)

Max Activating Token Index: 770

Text #2

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.6434. Min Act: -0.0000

Data Index: 1834885 (The Pile)

Max Activating Token Index: 115

Text #3

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.7005. Min Act: -0.0000

Data Index: 513726 (The Pile)

Max Activating Token Index: 746

Text #4

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4831. Min Act: -0.0000

Data Index: 1125284 (The Pile)

Max Activating Token Index: 943

Text #5

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4710. Min Act: -0.0000

Data Index: 845690 (The Pile)

Max Activating Token Index: 707

Text #6

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.5428. Min Act: -0.0000

Data Index: 1006092 (The Pile)

Max Activating Token Index: 470

Text #7

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.6961. Min Act: -0.0000

Data Index: 1166260 (The Pile)

Max Activating Token Index: 970

Text #8

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4729. Min Act: -0.0000

Data Index: 1586033 (The Pile)

Max Activating Token Index: 310

Text #9

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.5625. Min Act: -0.0000

Data Index: 1761520 (The Pile)

Max Activating Token Index: 964

Text #10

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.6807. Min Act: -0.0000

Data Index: 1022190 (The Pile)

Max Activating Token Index: 495

Text #11

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.5279. Min Act: -0.0000

Data Index: 697941 (The Pile)

Max Activating Token Index: 565

Text #12

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4416. Min Act: -0.0000

Data Index: 1307199 (The Pile)

Max Activating Token Index: 563

Text #13

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.3901. Min Act: -0.0000

Data Index: 593814 (The Pile)

Max Activating Token Index: 216

Text #14

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4772. Min Act: -0.0000

Data Index: 516555 (The Pile)

Max Activating Token Index: 300

Text #15

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4341. Min Act: -0.0000

Data Index: 123676 (The Pile)

Max Activating Token Index: 208

Text #16

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.3872. Min Act: -0.0000

Data Index: 1344983 (The Pile)

Max Activating Token Index: 595

Text #17

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4575. Min Act: -0.0000

Data Index: 603226 (The Pile)

Max Activating Token Index: 543

Text #18

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4546. Min Act: -0.0000

Data Index: 82802 (The Pile)

Max Activating Token Index: 870

Text #19

Max Range: 1.9508. Min Range: -1.9508

Max Act: 1.4356. Min Act: -0.0000

Data Index: 724094 (The Pile)

Max Activating Token Index: 292