Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2223 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.1011. Min Act: -0.0000

Data Index: 1312864 (The Pile)

Max Activating Token Index: 221

Text #1

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0554. Min Act: -0.0000

Data Index: 1706539 (The Pile)

Max Activating Token Index: 193

Text #2

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0841. Min Act: -0.0000

Data Index: 561313 (The Pile)

Max Activating Token Index: 681

Text #3

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0688. Min Act: -0.0000

Data Index: 1243603 (The Pile)

Max Activating Token Index: 389

Text #4

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0304. Min Act: -0.0000

Data Index: 502155 (The Pile)

Max Activating Token Index: 438

Text #5

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0630. Min Act: -0.0000

Data Index: 1433343 (The Pile)

Max Activating Token Index: 483

Text #6

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0592. Min Act: -0.0000

Data Index: 144395 (The Pile)

Max Activating Token Index: 500

Text #7

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0105. Min Act: -0.0000

Data Index: 301187 (The Pile)

Max Activating Token Index: 841

Text #8

Max Range: 1.1011. Min Range: -1.1011

Max Act: 1.0097. Min Act: -0.0000

Data Index: 450204 (The Pile)

Max Activating Token Index: 126

Text #9

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9634. Min Act: -0.0000

Data Index: 862744 (The Pile)

Max Activating Token Index: 470

Text #10

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9425. Min Act: -0.0000

Data Index: 906980 (The Pile)

Max Activating Token Index: 435

Text #11

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9302. Min Act: -0.0000

Data Index: 1955662 (The Pile)

Max Activating Token Index: 8

Text #12

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9729. Min Act: -0.0000

Data Index: 876812 (The Pile)

Max Activating Token Index: 286

Text #13

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9138. Min Act: -0.0000

Data Index: 92928 (The Pile)

Max Activating Token Index: 241

Text #14

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.8693. Min Act: -0.0000

Data Index: 748151 (The Pile)

Max Activating Token Index: 841

Text #15

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9073. Min Act: -0.0000

Data Index: 241630 (The Pile)

Max Activating Token Index: 402

Text #16

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9308. Min Act: -0.0000

Data Index: 966626 (The Pile)

Max Activating Token Index: 133

Text #17

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9837. Min Act: -0.0000

Data Index: 1313309 (The Pile)

Max Activating Token Index: 200

Text #18

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9202. Min Act: -0.0000

Data Index: 839232 (The Pile)

Max Activating Token Index: 316

Text #19

Max Range: 1.1011. Min Range: -1.1011

Max Act: 0.9251. Min Act: -0.0000

Data Index: 949921 (The Pile)

Max Activating Token Index: 504