Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2532 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.7452. Min Act: -0.0000

Data Index: 1704630 (The Pile)

Max Activating Token Index: 834

Text #1

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.7336. Min Act: -0.0000

Data Index: 416514 (The Pile)

Max Activating Token Index: 981

Text #2

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.6059. Min Act: -0.0000

Data Index: 121107 (The Pile)

Max Activating Token Index: 702

Text #3

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.6127. Min Act: -0.0000

Data Index: 1203765 (The Pile)

Max Activating Token Index: 281

Text #4

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.3578. Min Act: -0.0000

Data Index: 1208562 (The Pile)

Max Activating Token Index: 990

Text #5

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.3841. Min Act: -0.0000

Data Index: 1669671 (The Pile)

Max Activating Token Index: 731

Text #6

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.4644. Min Act: -0.0000

Data Index: 21043 (The Pile)

Max Activating Token Index: 890

Text #7

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.3143. Min Act: -0.0000

Data Index: 1944912 (The Pile)

Max Activating Token Index: 996

Text #8

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.3176. Min Act: -0.0000

Data Index: 269608 (The Pile)

Max Activating Token Index: 911

Text #9

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.2332. Min Act: -0.0000

Data Index: 119823 (The Pile)

Max Activating Token Index: 756

Text #10

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.0303. Min Act: -0.0000

Data Index: 748317 (The Pile)

Max Activating Token Index: 851

Text #11

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.2475. Min Act: -0.0000

Data Index: 1872985 (The Pile)

Max Activating Token Index: 307

Text #12

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.1738. Min Act: -0.0000

Data Index: 1605794 (The Pile)

Max Activating Token Index: 566

Text #13

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.3249. Min Act: -0.0000

Data Index: 1081639 (The Pile)

Max Activating Token Index: 1007

Text #14

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.2933. Min Act: -0.0000

Data Index: 63897 (The Pile)

Max Activating Token Index: 757

Text #15

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.3015. Min Act: -0.0000

Data Index: 1776240 (The Pile)

Max Activating Token Index: 102

Text #16

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.1103. Min Act: -0.0000

Data Index: 1338592 (The Pile)

Max Activating Token Index: 467

Text #17

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.1919. Min Act: -0.0000

Data Index: 383730 (The Pile)

Max Activating Token Index: 118

Text #18

Max Range: 1.7452. Min Range: -1.7452

Max Act: 1.1040. Min Act: -0.0000

Data Index: 992627 (The Pile)

Max Activating Token Index: 614

Text #19

Max Range: 1.7452. Min Range: -1.7452

Max Act: 0.9183. Min Act: -0.0000

Data Index: 326321 (The Pile)

Max Activating Token Index: 695