Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3275 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.9525. Min Act: -0.0000

Data Index: 1874181 (The Pile)

Max Activating Token Index: 228

Text #1

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6400. Min Act: -0.0000

Data Index: 484939 (The Pile)

Max Activating Token Index: 452

Text #2

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6296. Min Act: -0.0000

Data Index: 1168455 (The Pile)

Max Activating Token Index: 281

Text #3

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.7583. Min Act: -0.0000

Data Index: 1652645 (The Pile)

Max Activating Token Index: 683

Text #4

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6518. Min Act: -0.0000

Data Index: 1522580 (The Pile)

Max Activating Token Index: 233

Text #5

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.4512. Min Act: -0.0000

Data Index: 1584332 (The Pile)

Max Activating Token Index: 262

Text #6

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.8197. Min Act: -0.0000

Data Index: 1705118 (The Pile)

Max Activating Token Index: 259

Text #7

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6745. Min Act: -0.0000

Data Index: 1767592 (The Pile)

Max Activating Token Index: 191

Text #8

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6302. Min Act: -0.0000

Data Index: 1496105 (The Pile)

Max Activating Token Index: 122

Text #9

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.4362. Min Act: -0.0000

Data Index: 734353 (The Pile)

Max Activating Token Index: 145

Text #10

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.8274. Min Act: -0.0000

Data Index: 761354 (The Pile)

Max Activating Token Index: 683

Text #11

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6284. Min Act: -0.0000

Data Index: 489013 (The Pile)

Max Activating Token Index: 705

Text #12

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.4085. Min Act: -0.0000

Data Index: 1789791 (The Pile)

Max Activating Token Index: 439

Text #13

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.4107. Min Act: -0.0000

Data Index: 1603095 (The Pile)

Max Activating Token Index: 296

Text #14

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6087. Min Act: -0.0000

Data Index: 646757 (The Pile)

Max Activating Token Index: 108

Text #15

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.5976. Min Act: -0.0000

Data Index: 349895 (The Pile)

Max Activating Token Index: 298

Text #16

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.6034. Min Act: -0.0000

Data Index: 1758461 (The Pile)

Max Activating Token Index: 370

Text #17

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.5755. Min Act: -0.0000

Data Index: 1095128 (The Pile)

Max Activating Token Index: 426

Text #18

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.4012. Min Act: -0.0000

Data Index: 644845 (The Pile)

Max Activating Token Index: 159

Text #19

Max Range: 6.9525. Min Range: -6.9525

Max Act: 6.3974. Min Act: -0.0000

Data Index: 1159483 (The Pile)

Max Activating Token Index: 301