Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1336 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.8574. Min Act: -0.0000

Data Index: 101303 (The Pile)

Max Activating Token Index: 606

Text #1

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.6640. Min Act: -0.0000

Data Index: 1330682 (The Pile)

Max Activating Token Index: 126

Text #2

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.8240. Min Act: -0.0000

Data Index: 329560 (The Pile)

Max Activating Token Index: 810

Text #3

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.8230. Min Act: -0.0000

Data Index: 1046826 (The Pile)

Max Activating Token Index: 742

Text #4

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.9080. Min Act: -0.0000

Data Index: 1890354 (The Pile)

Max Activating Token Index: 238

Text #5

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.7025. Min Act: -0.0000

Data Index: 10647 (The Pile)

Max Activating Token Index: 828

Text #6

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.6154. Min Act: -0.0000

Data Index: 529369 (The Pile)

Max Activating Token Index: 9

Text #7

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.6603. Min Act: -0.0000

Data Index: 993117 (The Pile)

Max Activating Token Index: 570

Text #8

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.6891. Min Act: -0.0000

Data Index: 1084841 (The Pile)

Max Activating Token Index: 105

Text #9

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.7380. Min Act: -0.0000

Data Index: 664491 (The Pile)

Max Activating Token Index: 776

Text #10

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.7442. Min Act: -0.0000

Data Index: 1583848 (The Pile)

Max Activating Token Index: 124

Text #11

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.8237. Min Act: -0.0000

Data Index: 1078810 (The Pile)

Max Activating Token Index: 719

Text #12

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.5205. Min Act: -0.0000

Data Index: 1681443 (The Pile)

Max Activating Token Index: 948

Text #13

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.6279. Min Act: -0.0000

Data Index: 456540 (The Pile)

Max Activating Token Index: 106

Text #14

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.6487. Min Act: -0.0000

Data Index: 953237 (The Pile)

Max Activating Token Index: 483

Text #15

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.7064. Min Act: -0.0000

Data Index: 1638096 (The Pile)

Max Activating Token Index: 475

Text #16

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.6276. Min Act: -0.0000

Data Index: 754056 (The Pile)

Max Activating Token Index: 819

Text #17

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.5185. Min Act: -0.0000

Data Index: 558450 (The Pile)

Max Activating Token Index: 709

Text #18

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.8159. Min Act: -0.0000

Data Index: 849105 (The Pile)

Max Activating Token Index: 86

Text #19

Max Range: 1.9080. Min Range: -1.9080

Max Act: 1.7121. Min Act: -0.0000

Data Index: 1346092 (The Pile)

Max Activating Token Index: 36