Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1379 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 7.4661. Min Range: -7.4661

Max Act: 7.4661. Min Act: -0.0000

Data Index: 237984 (The Pile)

Max Activating Token Index: 72

Text #1

Max Range: 7.4661. Min Range: -7.4661

Max Act: 7.4661. Min Act: -0.0000

Data Index: 480510 (The Pile)

Max Activating Token Index: 72

Text #2

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.3968. Min Act: -0.0000

Data Index: 1930092 (The Pile)

Max Activating Token Index: 669

Text #3

Max Range: 7.4661. Min Range: -7.4661

Max Act: 7.0332. Min Act: -0.0000

Data Index: 1889425 (The Pile)

Max Activating Token Index: 904

Text #4

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.6399. Min Act: -0.0000

Data Index: 23703 (The Pile)

Max Activating Token Index: 982

Text #5

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.7179. Min Act: -0.0000

Data Index: 1883945 (The Pile)

Max Activating Token Index: 428

Text #6

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.4806. Min Act: -0.0000

Data Index: 1239556 (The Pile)

Max Activating Token Index: 829

Text #7

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.3987. Min Act: -0.0000

Data Index: 1588330 (The Pile)

Max Activating Token Index: 960

Text #8

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.5344. Min Act: -0.0000

Data Index: 1103152 (The Pile)

Max Activating Token Index: 542

Text #9

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.4107. Min Act: -0.0000

Data Index: 61057 (The Pile)

Max Activating Token Index: 851

Text #10

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.4809. Min Act: -0.0000

Data Index: 1559986 (The Pile)

Max Activating Token Index: 81

Text #11

Max Range: 7.4661. Min Range: -7.4661

Max Act: 5.8683. Min Act: -0.0000

Data Index: 945458 (The Pile)

Max Activating Token Index: 182

Text #12

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.2042. Min Act: -0.0000

Data Index: 96167 (The Pile)

Max Activating Token Index: 302

Text #13

Max Range: 7.4661. Min Range: -7.4661

Max Act: 5.9719. Min Act: -0.0000

Data Index: 287025 (The Pile)

Max Activating Token Index: 730

Text #14

Max Range: 7.4661. Min Range: -7.4661

Max Act: 5.8389. Min Act: -0.0000

Data Index: 1398057 (The Pile)

Max Activating Token Index: 635

Text #15

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.3631. Min Act: -0.0000

Data Index: 1999959 (The Pile)

Max Activating Token Index: 560

Text #16

Max Range: 7.4661. Min Range: -7.4661

Max Act: 6.2636. Min Act: -0.0000

Data Index: 103145 (The Pile)

Max Activating Token Index: 74

Text #17

Max Range: 7.4661. Min Range: -7.4661

Max Act: 5.8490. Min Act: -0.0000

Data Index: 1258002 (The Pile)

Max Activating Token Index: 223

Text #18

Max Range: 7.4661. Min Range: -7.4661

Max Act: 5.7461. Min Act: -0.0000

Data Index: 359483 (The Pile)

Max Activating Token Index: 99

Text #19

Max Range: 7.4661. Min Range: -7.4661

Max Act: 5.6555. Min Act: -0.0000

Data Index: 485772 (The Pile)

Max Activating Token Index: 107