Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2074 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.5284. Min Range: -3.5284

Max Act: 3.5284. Min Act: -0.0000

Data Index: 314251 (The Pile)

Max Activating Token Index: 384

Text #1

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.7054. Min Act: -0.0000

Data Index: 1696788 (The Pile)

Max Activating Token Index: 92

Text #2

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.7163. Min Act: 0.0000

Data Index: 150722 (The Pile)

Max Activating Token Index: 678

Text #3

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.3310. Min Act: -0.0000

Data Index: 1320013 (The Pile)

Max Activating Token Index: 584

Text #4

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.5139. Min Act: -0.0000

Data Index: 1289896 (The Pile)

Max Activating Token Index: 565

Text #5

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.5324. Min Act: -0.0000

Data Index: 1416397 (The Pile)

Max Activating Token Index: 714

Text #6

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.6196. Min Act: -0.0000

Data Index: 1787947 (The Pile)

Max Activating Token Index: 92

Text #7

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.5670. Min Act: -0.0000

Data Index: 324957 (The Pile)

Max Activating Token Index: 674

Text #8

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.4291. Min Act: -0.0000

Data Index: 271220 (The Pile)

Max Activating Token Index: 116

Text #9

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.2948. Min Act: -0.0000

Data Index: 376721 (The Pile)

Max Activating Token Index: 1016

Text #10

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.2428. Min Act: -0.0000

Data Index: 1172757 (The Pile)

Max Activating Token Index: 464

Text #11

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.0065. Min Act: -0.0000

Data Index: 517701 (The Pile)

Max Activating Token Index: 139

Text #12

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.3368. Min Act: -0.0000

Data Index: 310812 (The Pile)

Max Activating Token Index: 64

Text #13

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.3304. Min Act: -0.0000

Data Index: 1697419 (The Pile)

Max Activating Token Index: 673

Text #14

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.1744. Min Act: -0.0000

Data Index: 1006313 (The Pile)

Max Activating Token Index: 258

Text #15

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.1351. Min Act: -0.0000

Data Index: 1789986 (The Pile)

Max Activating Token Index: 930

Text #16

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.1458. Min Act: -0.0000

Data Index: 112289 (The Pile)

Max Activating Token Index: 179

Text #17

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.0056. Min Act: -0.0000

Data Index: 918461 (The Pile)

Max Activating Token Index: 231

Text #18

Max Range: 3.5284. Min Range: -3.5284

Max Act: 1.9994. Min Act: -0.0000

Data Index: 826474 (The Pile)

Max Activating Token Index: 63

Text #19

Max Range: 3.5284. Min Range: -3.5284

Max Act: 2.0136. Min Act: -0.0000

Data Index: 183233 (The Pile)

Max Activating Token Index: 861