Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 174 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7954. Min Range: -3.7954

Max Act: 3.7954. Min Act: -0.0000

Data Index: 168818 (The Pile)

Max Activating Token Index: 215

Text #1

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.9488. Min Act: -0.0000

Data Index: 1724254 (The Pile)

Max Activating Token Index: 396

Text #2

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.6628. Min Act: -0.0000

Data Index: 1491704 (The Pile)

Max Activating Token Index: 201

Text #3

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.7400. Min Act: -0.0000

Data Index: 1846804 (The Pile)

Max Activating Token Index: 438

Text #4

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.7003. Min Act: -0.0000

Data Index: 1157332 (The Pile)

Max Activating Token Index: 282

Text #5

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.3717. Min Act: -0.0000

Data Index: 1289130 (The Pile)

Max Activating Token Index: 8

Text #6

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.3035. Min Act: -0.0000

Data Index: 90704 (The Pile)

Max Activating Token Index: 173

Text #7

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.2025. Min Act: -0.0000

Data Index: 1032253 (The Pile)

Max Activating Token Index: 371

Text #8

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.3600. Min Act: -0.0000

Data Index: 1288631 (The Pile)

Max Activating Token Index: 83

Text #9

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.1258. Min Act: -0.0000

Data Index: 1576403 (The Pile)

Max Activating Token Index: 528

Text #10

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.2526. Min Act: -0.0000

Data Index: 181667 (The Pile)

Max Activating Token Index: 587

Text #11

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.4176. Min Act: -0.0000

Data Index: 494202 (The Pile)

Max Activating Token Index: 597

Text #12

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.0173. Min Act: -0.0000

Data Index: 49945 (The Pile)

Max Activating Token Index: 479

Text #13

Max Range: 3.7954. Min Range: -3.7954

Max Act: 1.8969. Min Act: -0.0000

Data Index: 1997045 (The Pile)

Max Activating Token Index: 179

Text #14

Max Range: 3.7954. Min Range: -3.7954

Max Act: 1.6901. Min Act: -0.0000

Data Index: 1820285 (The Pile)

Max Activating Token Index: 459

Text #15

Max Range: 3.7954. Min Range: -3.7954

Max Act: 2.0640. Min Act: -0.0000

Data Index: 428073 (The Pile)

Max Activating Token Index: 349

Text #16

Max Range: 3.7954. Min Range: -3.7954

Max Act: 1.9029. Min Act: -0.0000

Data Index: 1452561 (The Pile)

Max Activating Token Index: 463

Text #17

Max Range: 3.7954. Min Range: -3.7954

Max Act: 1.9225. Min Act: -0.0000

Data Index: 1915748 (The Pile)

Max Activating Token Index: 561

Text #18

Max Range: 3.7954. Min Range: -3.7954

Max Act: 1.8927. Min Act: -0.0000

Data Index: 759412 (The Pile)

Max Activating Token Index: 280

Text #19

Max Range: 3.7954. Min Range: -3.7954

Max Act: 1.8844. Min Act: -0.0000

Data Index: 954253 (The Pile)

Max Activating Token Index: 280