Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1462 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.5703. Min Act: -0.0000

Data Index: 1561238 (The Pile)

Max Activating Token Index: 197

Text #1

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.4070. Min Act: -0.0000

Data Index: 1297246 (The Pile)

Max Activating Token Index: 188

Text #2

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.2295. Min Act: -0.0000

Data Index: 1047990 (The Pile)

Max Activating Token Index: 86

Text #3

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.1276. Min Act: -0.0000

Data Index: 868406 (The Pile)

Max Activating Token Index: 280

Text #4

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.1624. Min Act: -0.0000

Data Index: 1810794 (The Pile)

Max Activating Token Index: 287

Text #5

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.1807. Min Act: -0.0000

Data Index: 473741 (The Pile)

Max Activating Token Index: 38

Text #6

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0869. Min Act: -0.0000

Data Index: 1931249 (The Pile)

Max Activating Token Index: 77

Text #7

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0662. Min Act: -0.0000

Data Index: 125847 (The Pile)

Max Activating Token Index: 630

Text #8

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0808. Min Act: -0.0000

Data Index: 1267193 (The Pile)

Max Activating Token Index: 72

Text #9

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0345. Min Act: -0.0000

Data Index: 349306 (The Pile)

Max Activating Token Index: 95

Text #10

Max Range: 1.5703. Min Range: -1.5703

Max Act: 0.9881. Min Act: -0.0000

Data Index: 346999 (The Pile)

Max Activating Token Index: 1001

Text #11

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0125. Min Act: -0.0000

Data Index: 270142 (The Pile)

Max Activating Token Index: 127

Text #12

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0407. Min Act: -0.0000

Data Index: 1532867 (The Pile)

Max Activating Token Index: 253

Text #13

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0238. Min Act: -0.0000

Data Index: 1266518 (The Pile)

Max Activating Token Index: 84

Text #14

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0259. Min Act: -0.0000

Data Index: 1186381 (The Pile)

Max Activating Token Index: 149

Text #15

Max Range: 1.5703. Min Range: -1.5703

Max Act: 0.9422. Min Act: -0.0000

Data Index: 1057148 (The Pile)

Max Activating Token Index: 307

Text #16

Max Range: 1.5703. Min Range: -1.5703

Max Act: 0.9943. Min Act: -0.0000

Data Index: 654011 (The Pile)

Max Activating Token Index: 271

Text #17

Max Range: 1.5703. Min Range: -1.5703

Max Act: 1.0080. Min Act: -0.0000

Data Index: 1089620 (The Pile)

Max Activating Token Index: 47

Text #18

Max Range: 1.5703. Min Range: -1.5703

Max Act: 0.9638. Min Act: -0.0000

Data Index: 627569 (The Pile)

Max Activating Token Index: 505

Text #19

Max Range: 1.5703. Min Range: -1.5703

Max Act: 0.9359. Min Act: -0.0000

Data Index: 992404 (The Pile)

Max Activating Token Index: 596