Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1535 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.8193. Min Act: -0.0000

Data Index: 1323006 (The Pile)

Max Activating Token Index: 534

Text #1

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.6060. Min Act: -0.0000

Data Index: 157930 (The Pile)

Max Activating Token Index: 885

Text #2

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.4470. Min Act: -0.0000

Data Index: 96300 (The Pile)

Max Activating Token Index: 856

Text #3

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.4440. Min Act: -0.0000

Data Index: 1527151 (The Pile)

Max Activating Token Index: 453

Text #4

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.2898. Min Act: -0.0000

Data Index: 1298043 (The Pile)

Max Activating Token Index: 739

Text #5

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.3282. Min Act: -0.0000

Data Index: 340680 (The Pile)

Max Activating Token Index: 18

Text #6

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.3183. Min Act: -0.0000

Data Index: 776794 (The Pile)

Max Activating Token Index: 382

Text #7

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0594. Min Act: -0.0000

Data Index: 1824863 (The Pile)

Max Activating Token Index: 676

Text #8

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.1013. Min Act: -0.0000

Data Index: 1105086 (The Pile)

Max Activating Token Index: 739

Text #9

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0250. Min Act: 0.0000

Data Index: 1897330 (The Pile)

Max Activating Token Index: 102

Text #10

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0174. Min Act: -0.0000

Data Index: 1036400 (The Pile)

Max Activating Token Index: 281

Text #11

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0129. Min Act: -0.0000

Data Index: 1953158 (The Pile)

Max Activating Token Index: 340

Text #12

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0085. Min Act: 0.0000

Data Index: 412325 (The Pile)

Max Activating Token Index: 999

Text #13

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0603. Min Act: -0.0000

Data Index: 964887 (The Pile)

Max Activating Token Index: 58

Text #14

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0703. Min Act: -0.0000

Data Index: 1439754 (The Pile)

Max Activating Token Index: 437

Text #15

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0476. Min Act: -0.0000

Data Index: 919206 (The Pile)

Max Activating Token Index: 542

Text #16

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0352. Min Act: -0.0000

Data Index: 1046886 (The Pile)

Max Activating Token Index: 281

Text #17

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0840. Min Act: -0.0000

Data Index: 1609461 (The Pile)

Max Activating Token Index: 412

Text #18

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0052. Min Act: -0.0000

Data Index: 566334 (The Pile)

Max Activating Token Index: 414

Text #19

Max Range: 1.8193. Min Range: -1.8193

Max Act: 1.0060. Min Act: -0.0000

Data Index: 1702241 (The Pile)

Max Activating Token Index: 158