Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1102 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1616. Min Act: -0.0000

Data Index: 1568481 (The Pile)

Max Activating Token Index: 943

Text #1

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1228. Min Act: -0.0000

Data Index: 1768449 (The Pile)

Max Activating Token Index: 878

Text #2

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1462. Min Act: -0.0000

Data Index: 879095 (The Pile)

Max Activating Token Index: 164

Text #3

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1433. Min Act: -0.0000

Data Index: 794390 (The Pile)

Max Activating Token Index: 435

Text #4

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1505. Min Act: -0.0000

Data Index: 1472363 (The Pile)

Max Activating Token Index: 182

Text #5

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1204. Min Act: -0.0000

Data Index: 1810413 (The Pile)

Max Activating Token Index: 196

Text #6

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1244. Min Act: -0.0000

Data Index: 1352760 (The Pile)

Max Activating Token Index: 169

Text #7

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1322. Min Act: -0.0000

Data Index: 864704 (The Pile)

Max Activating Token Index: 202

Text #8

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1185. Min Act: -0.0000

Data Index: 1327842 (The Pile)

Max Activating Token Index: 749

Text #9

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1320. Min Act: -0.0000

Data Index: 385243 (The Pile)

Max Activating Token Index: 322

Text #10

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1290. Min Act: -0.0000

Data Index: 213102 (The Pile)

Max Activating Token Index: 882

Text #11

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1328. Min Act: -0.0000

Data Index: 527539 (The Pile)

Max Activating Token Index: 177

Text #12

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1283. Min Act: -0.0000

Data Index: 1122664 (The Pile)

Max Activating Token Index: 537

Text #13

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1154. Min Act: -0.0000

Data Index: 756781 (The Pile)

Max Activating Token Index: 878

Text #14

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1240. Min Act: -0.0000

Data Index: 713256 (The Pile)

Max Activating Token Index: 762

Text #15

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1199. Min Act: -0.0000

Data Index: 163074 (The Pile)

Max Activating Token Index: 779

Text #16

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1039. Min Act: -0.0000

Data Index: 426764 (The Pile)

Max Activating Token Index: 980

Text #17

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1151. Min Act: -0.0000

Data Index: 97388 (The Pile)

Max Activating Token Index: 687

Text #18

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1185. Min Act: -0.0000

Data Index: 1834783 (The Pile)

Max Activating Token Index: 338

Text #19

Max Range: 0.1616. Min Range: -0.1616

Max Act: 0.1196. Min Act: -0.0000

Data Index: 793772 (The Pile)

Max Activating Token Index: 240