Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3661 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7643. Min Range: -3.7643

Max Act: 3.7643. Min Act: -0.0000

Data Index: 266265 (The Pile)

Max Activating Token Index: 590

Text #1

Max Range: 3.7643. Min Range: -3.7643

Max Act: 3.3486. Min Act: -0.0000

Data Index: 695603 (The Pile)

Max Activating Token Index: 271

Text #2

Max Range: 3.7643. Min Range: -3.7643

Max Act: 3.0937. Min Act: -0.0000

Data Index: 1861585 (The Pile)

Max Activating Token Index: 1021

Text #3

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.8855. Min Act: -0.0000

Data Index: 21564 (The Pile)

Max Activating Token Index: 299

Text #4

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.9123. Min Act: -0.0000

Data Index: 1044451 (The Pile)

Max Activating Token Index: 643

Text #5

Max Range: 3.7643. Min Range: -3.7643

Max Act: 3.0140. Min Act: -0.0000

Data Index: 1071651 (The Pile)

Max Activating Token Index: 241

Text #6

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.6339. Min Act: -0.0000

Data Index: 160562 (The Pile)

Max Activating Token Index: 521

Text #7

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.7381. Min Act: -0.0000

Data Index: 1793130 (The Pile)

Max Activating Token Index: 587

Text #8

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.7179. Min Act: -0.0000

Data Index: 1827561 (The Pile)

Max Activating Token Index: 538

Text #9

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.5331. Min Act: -0.0000

Data Index: 199341 (The Pile)

Max Activating Token Index: 262

Text #10

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.5387. Min Act: -0.0000

Data Index: 1020384 (The Pile)

Max Activating Token Index: 204

Text #11

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.4272. Min Act: -0.0000

Data Index: 165165 (The Pile)

Max Activating Token Index: 544

Text #12

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.5259. Min Act: -0.0000

Data Index: 204346 (The Pile)

Max Activating Token Index: 48

Text #13

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.3897. Min Act: -0.0000

Data Index: 1996649 (The Pile)

Max Activating Token Index: 720

Text #14

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.3900. Min Act: -0.0000

Data Index: 379574 (The Pile)

Max Activating Token Index: 404

Text #15

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.4990. Min Act: -0.0000

Data Index: 122787 (The Pile)

Max Activating Token Index: 249

Text #16

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.5526. Min Act: -0.0000

Data Index: 1741160 (The Pile)

Max Activating Token Index: 912

Text #17

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.4440. Min Act: -0.0000

Data Index: 956938 (The Pile)

Max Activating Token Index: 247

Text #18

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.3248. Min Act: -0.0000

Data Index: 679216 (The Pile)

Max Activating Token Index: 422

Text #19

Max Range: 3.7643. Min Range: -3.7643

Max Act: 2.3210. Min Act: -0.0000

Data Index: 747873 (The Pile)

Max Activating Token Index: 782