Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4591 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.5325. Min Act: -0.0000

Data Index: 369361 (The Pile)

Max Activating Token Index: 514

Text #1

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.4435. Min Act: -0.0000

Data Index: 1048297 (The Pile)

Max Activating Token Index: 396

Text #2

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.4538. Min Act: -0.0000

Data Index: 190939 (The Pile)

Max Activating Token Index: 401

Text #3

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.4462. Min Act: -0.0000

Data Index: 1480111 (The Pile)

Max Activating Token Index: 614

Text #4

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3983. Min Act: -0.0000

Data Index: 1026510 (The Pile)

Max Activating Token Index: 743

Text #5

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.4100. Min Act: -0.0000

Data Index: 1020912 (The Pile)

Max Activating Token Index: 409

Text #6

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.4027. Min Act: -0.0000

Data Index: 845869 (The Pile)

Max Activating Token Index: 766

Text #7

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3997. Min Act: -0.0000

Data Index: 896832 (The Pile)

Max Activating Token Index: 659

Text #8

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3814. Min Act: -0.0000

Data Index: 430744 (The Pile)

Max Activating Token Index: 728

Text #9

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3862. Min Act: -0.0000

Data Index: 1582712 (The Pile)

Max Activating Token Index: 950

Text #10

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3941. Min Act: -0.0000

Data Index: 405748 (The Pile)

Max Activating Token Index: 92

Text #11

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.4207. Min Act: -0.0000

Data Index: 1897268 (The Pile)

Max Activating Token Index: 758

Text #12

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3552. Min Act: -0.0000

Data Index: 1204896 (The Pile)

Max Activating Token Index: 622

Text #13

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3205. Min Act: -0.0000

Data Index: 778272 (The Pile)

Max Activating Token Index: 984

Text #14

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3336. Min Act: -0.0000

Data Index: 1155276 (The Pile)

Max Activating Token Index: 177

Text #15

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3625. Min Act: -0.0000

Data Index: 425744 (The Pile)

Max Activating Token Index: 70

Text #16

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3322. Min Act: -0.0000

Data Index: 1371902 (The Pile)

Max Activating Token Index: 614

Text #17

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3120. Min Act: -0.0000

Data Index: 1512726 (The Pile)

Max Activating Token Index: 830

Text #18

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3366. Min Act: -0.0000

Data Index: 1468102 (The Pile)

Max Activating Token Index: 655

Text #19

Max Range: 0.5325. Min Range: -0.5325

Max Act: 0.3446. Min Act: -0.0000

Data Index: 1597457 (The Pile)

Max Activating Token Index: 746