Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2324 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.1078. Min Act: -0.0000

Data Index: 1139991 (The Pile)

Max Activating Token Index: 1023

Text #1

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.1045. Min Act: -0.0000

Data Index: 539610 (The Pile)

Max Activating Token Index: 28

Text #2

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.1036. Min Act: -0.0000

Data Index: 1537818 (The Pile)

Max Activating Token Index: 66

Text #3

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.1005. Min Act: -0.0000

Data Index: 1768621 (The Pile)

Max Activating Token Index: 50

Text #4

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.1022. Min Act: -0.0000

Data Index: 1818142 (The Pile)

Max Activating Token Index: 71

Text #5

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0978. Min Act: -0.0000

Data Index: 1080361 (The Pile)

Max Activating Token Index: 202

Text #6

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0973. Min Act: -0.0000

Data Index: 373296 (The Pile)

Max Activating Token Index: 17

Text #7

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0967. Min Act: -0.0000

Data Index: 1214682 (The Pile)

Max Activating Token Index: 498

Text #8

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0962. Min Act: -0.0000

Data Index: 1796041 (The Pile)

Max Activating Token Index: 14

Text #9

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0943. Min Act: -0.0000

Data Index: 258865 (The Pile)

Max Activating Token Index: 40

Text #10

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0941. Min Act: -0.0000

Data Index: 1469562 (The Pile)

Max Activating Token Index: 36

Text #11

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0896. Min Act: -0.0000

Data Index: 152297 (The Pile)

Max Activating Token Index: 980

Text #12

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0963. Min Act: -0.0000

Data Index: 1100599 (The Pile)

Max Activating Token Index: 107

Text #13

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0927. Min Act: -0.0000

Data Index: 1764434 (The Pile)

Max Activating Token Index: 173

Text #14

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0925. Min Act: -0.0000

Data Index: 493013 (The Pile)

Max Activating Token Index: 159

Text #15

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0921. Min Act: -0.0000

Data Index: 1485261 (The Pile)

Max Activating Token Index: 124

Text #16

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0916. Min Act: -0.0000

Data Index: 1262445 (The Pile)

Max Activating Token Index: 274

Text #17

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0910. Min Act: -0.0000

Data Index: 208755 (The Pile)

Max Activating Token Index: 6

Text #18

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0908. Min Act: -0.0000

Data Index: 1166319 (The Pile)

Max Activating Token Index: 169

Text #19

Max Range: 0.1078. Min Range: -0.1078

Max Act: 0.0937. Min Act: -0.0000

Data Index: 943281 (The Pile)

Max Activating Token Index: 6