Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1722 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1464. Min Act: -0.0000

Data Index: 1189109 (The Pile)

Max Activating Token Index: 766

Text #1

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1263. Min Act: -0.0000

Data Index: 1133950 (The Pile)

Max Activating Token Index: 149

Text #2

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1308. Min Act: -0.0000

Data Index: 1027282 (The Pile)

Max Activating Token Index: 371

Text #3

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1222. Min Act: -0.0000

Data Index: 34483 (The Pile)

Max Activating Token Index: 527

Text #4

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1092. Min Act: -0.0000

Data Index: 983925 (The Pile)

Max Activating Token Index: 459

Text #5

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1174. Min Act: -0.0000

Data Index: 496789 (The Pile)

Max Activating Token Index: 259

Text #6

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.0993. Min Act: -0.0000

Data Index: 1669908 (The Pile)

Max Activating Token Index: 247

Text #7

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1096. Min Act: -0.0000

Data Index: 1777594 (The Pile)

Max Activating Token Index: 717

Text #8

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1027. Min Act: -0.0000

Data Index: 106709 (The Pile)

Max Activating Token Index: 716

Text #9

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1005. Min Act: -0.0000

Data Index: 1056308 (The Pile)

Max Activating Token Index: 557

Text #10

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1031. Min Act: -0.0000

Data Index: 1056850 (The Pile)

Max Activating Token Index: 206

Text #11

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1008. Min Act: -0.0000

Data Index: 1554844 (The Pile)

Max Activating Token Index: 315

Text #12

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1025. Min Act: -0.0000

Data Index: 1772436 (The Pile)

Max Activating Token Index: 116

Text #13

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.0963. Min Act: -0.0000

Data Index: 1960605 (The Pile)

Max Activating Token Index: 702

Text #14

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1013. Min Act: -0.0000

Data Index: 131930 (The Pile)

Max Activating Token Index: 582

Text #15

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.0996. Min Act: -0.0000

Data Index: 1420268 (The Pile)

Max Activating Token Index: 605

Text #16

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.1065. Min Act: -0.0000

Data Index: 826924 (The Pile)

Max Activating Token Index: 256

Text #17

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.0914. Min Act: -0.0000

Data Index: 1339221 (The Pile)

Max Activating Token Index: 187

Text #18

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.0956. Min Act: -0.0000

Data Index: 1621152 (The Pile)

Max Activating Token Index: 375

Text #19

Max Range: 0.1464. Min Range: -0.1464

Max Act: 0.0926. Min Act: -0.0000

Data Index: 1112306 (The Pile)

Max Activating Token Index: 194