Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2853 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2834. Min Range: -1.2834

Max Act: 1.2834. Min Act: -0.0000

Data Index: 789981 (The Pile)

Max Activating Token Index: 678

Text #1

Max Range: 1.2834. Min Range: -1.2834

Max Act: 1.1300. Min Act: -0.0000

Data Index: 1055718 (The Pile)

Max Activating Token Index: 377

Text #2

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.8952. Min Act: -0.0000

Data Index: 936713 (The Pile)

Max Activating Token Index: 330

Text #3

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.8964. Min Act: -0.0000

Data Index: 874746 (The Pile)

Max Activating Token Index: 498

Text #4

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7806. Min Act: -0.0000

Data Index: 315903 (The Pile)

Max Activating Token Index: 155

Text #5

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.8783. Min Act: -0.0000

Data Index: 1015849 (The Pile)

Max Activating Token Index: 192

Text #6

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.8459. Min Act: -0.0000

Data Index: 1542631 (The Pile)

Max Activating Token Index: 381

Text #7

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7957. Min Act: -0.0000

Data Index: 1818651 (The Pile)

Max Activating Token Index: 349

Text #8

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7409. Min Act: -0.0000

Data Index: 420237 (The Pile)

Max Activating Token Index: 642

Text #9

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7544. Min Act: -0.0000

Data Index: 509402 (The Pile)

Max Activating Token Index: 324

Text #10

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7524. Min Act: -0.0000

Data Index: 844554 (The Pile)

Max Activating Token Index: 313

Text #11

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7190. Min Act: -0.0000

Data Index: 1441209 (The Pile)

Max Activating Token Index: 69

Text #12

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7311. Min Act: -0.0000

Data Index: 1971172 (The Pile)

Max Activating Token Index: 289

Text #13

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.6831. Min Act: -0.0000

Data Index: 494544 (The Pile)

Max Activating Token Index: 1004

Text #14

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.7068. Min Act: -0.0000

Data Index: 806812 (The Pile)

Max Activating Token Index: 489

Text #15

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.6743. Min Act: -0.0000

Data Index: 1615305 (The Pile)

Max Activating Token Index: 877

Text #16

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.6140. Min Act: -0.0000

Data Index: 1650274 (The Pile)

Max Activating Token Index: 705

Text #17

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.6666. Min Act: -0.0000

Data Index: 1118179 (The Pile)

Max Activating Token Index: 132

Text #18

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.6494. Min Act: -0.0000

Data Index: 1060980 (The Pile)

Max Activating Token Index: 621

Text #19

Max Range: 1.2834. Min Range: -1.2834

Max Act: 0.6885. Min Act: -0.0000

Data Index: 1796466 (The Pile)

Max Activating Token Index: 539