Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 703 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.8157. Min Act: -0.0000

Data Index: 1170564 (The Pile)

Max Activating Token Index: 147

Text #1

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.8204. Min Act: -0.0000

Data Index: 881265 (The Pile)

Max Activating Token Index: 423

Text #2

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.6462. Min Act: -0.0000

Data Index: 1087366 (The Pile)

Max Activating Token Index: 981

Text #3

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.5282. Min Act: -0.0000

Data Index: 1447416 (The Pile)

Max Activating Token Index: 957

Text #4

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.5330. Min Act: -0.0000

Data Index: 83366 (The Pile)

Max Activating Token Index: 723

Text #5

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.4276. Min Act: -0.0000

Data Index: 1196316 (The Pile)

Max Activating Token Index: 752

Text #6

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.3579. Min Act: -0.0000

Data Index: 1344553 (The Pile)

Max Activating Token Index: 148

Text #7

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.3581. Min Act: -0.0000

Data Index: 1219768 (The Pile)

Max Activating Token Index: 75

Text #8

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.3823. Min Act: -0.0000

Data Index: 1457589 (The Pile)

Max Activating Token Index: 249

Text #9

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2788. Min Act: -0.0000

Data Index: 971794 (The Pile)

Max Activating Token Index: 884

Text #10

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.3223. Min Act: -0.0000

Data Index: 1127600 (The Pile)

Max Activating Token Index: 566

Text #11

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2941. Min Act: -0.0000

Data Index: 1797730 (The Pile)

Max Activating Token Index: 617

Text #12

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.3114. Min Act: -0.0000

Data Index: 252806 (The Pile)

Max Activating Token Index: 148

Text #13

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2540. Min Act: -0.0000

Data Index: 280817 (The Pile)

Max Activating Token Index: 82

Text #14

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2650. Min Act: -0.0000

Data Index: 463198 (The Pile)

Max Activating Token Index: 299

Text #15

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2833. Min Act: -0.0000

Data Index: 415291 (The Pile)

Max Activating Token Index: 346

Text #16

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2998. Min Act: -0.0000

Data Index: 60159 (The Pile)

Max Activating Token Index: 953

Text #17

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.3074. Min Act: -0.0000

Data Index: 48483 (The Pile)

Max Activating Token Index: 336

Text #18

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2900. Min Act: -0.0000

Data Index: 1131541 (The Pile)

Max Activating Token Index: 812

Text #19

Max Range: 0.8204. Min Range: -0.8204

Max Act: 0.2605. Min Act: -0.0000

Data Index: 1407027 (The Pile)

Max Activating Token Index: 208