Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 718 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0397. Min Act: -0.0000

Data Index: 1932914 (The Pile)

Max Activating Token Index: 711

Text #1

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0320. Min Act: -0.0000

Data Index: 870237 (The Pile)

Max Activating Token Index: 909

Text #2

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0266. Min Act: -0.0000

Data Index: 1083958 (The Pile)

Max Activating Token Index: 935

Text #3

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0186. Min Act: -0.0000

Data Index: 690478 (The Pile)

Max Activating Token Index: 976

Text #4

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0264. Min Act: -0.0000

Data Index: 1709555 (The Pile)

Max Activating Token Index: 166

Text #5

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0164. Min Act: -0.0000

Data Index: 1086126 (The Pile)

Max Activating Token Index: 857

Text #6

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0231. Min Act: -0.0000

Data Index: 7552 (The Pile)

Max Activating Token Index: 976

Text #7

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0252. Min Act: -0.0000

Data Index: 142223 (The Pile)

Max Activating Token Index: 663

Text #8

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0224. Min Act: -0.0000

Data Index: 341313 (The Pile)

Max Activating Token Index: 981

Text #9

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0257. Min Act: -0.0000

Data Index: 1607098 (The Pile)

Max Activating Token Index: 795

Text #10

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0205. Min Act: -0.0000

Data Index: 1253005 (The Pile)

Max Activating Token Index: 997

Text #11

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0132. Min Act: -0.0000

Data Index: 95646 (The Pile)

Max Activating Token Index: 1007

Text #12

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0215. Min Act: -0.0000

Data Index: 664352 (The Pile)

Max Activating Token Index: 973

Text #13

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0215. Min Act: -0.0000

Data Index: 556194 (The Pile)

Max Activating Token Index: 712

Text #14

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0240. Min Act: -0.0000

Data Index: 1712279 (The Pile)

Max Activating Token Index: 14

Text #15

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0185. Min Act: -0.0000

Data Index: 1906281 (The Pile)

Max Activating Token Index: 643

Text #16

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0192. Min Act: -0.0000

Data Index: 1275629 (The Pile)

Max Activating Token Index: 974

Text #17

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0229. Min Act: -0.0000

Data Index: 1428035 (The Pile)

Max Activating Token Index: 919

Text #18

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0216. Min Act: -0.0000

Data Index: 476502 (The Pile)

Max Activating Token Index: 1019

Text #19

Max Range: 0.0397. Min Range: -0.0397

Max Act: 0.0186. Min Act: -0.0000

Data Index: 61753 (The Pile)

Max Activating Token Index: 812