Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3380 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.1626. Min Act: -0.0000

Data Index: 1409355 (The Pile)

Max Activating Token Index: 378

Text #1

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.1522. Min Act: -0.0000

Data Index: 401585 (The Pile)

Max Activating Token Index: 324

Text #2

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.1411. Min Act: -0.0000

Data Index: 1992273 (The Pile)

Max Activating Token Index: 492

Text #3

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.0602. Min Act: -0.0000

Data Index: 958697 (The Pile)

Max Activating Token Index: 196

Text #4

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.1913. Min Act: -0.0000

Data Index: 741179 (The Pile)

Max Activating Token Index: 645

Text #5

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.1746. Min Act: -0.0000

Data Index: 509372 (The Pile)

Max Activating Token Index: 862

Text #6

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.1114. Min Act: -0.0000

Data Index: 1165836 (The Pile)

Max Activating Token Index: 400

Text #7

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.1104. Min Act: -0.0000

Data Index: 1098702 (The Pile)

Max Activating Token Index: 154

Text #8

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.0449. Min Act: -0.0000

Data Index: 171425 (The Pile)

Max Activating Token Index: 797

Text #9

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.0784. Min Act: -0.0000

Data Index: 404073 (The Pile)

Max Activating Token Index: 451

Text #10

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.0554. Min Act: -0.0000

Data Index: 1649864 (The Pile)

Max Activating Token Index: 74

Text #11

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.0409. Min Act: -0.0000

Data Index: 1265257 (The Pile)

Max Activating Token Index: 312

Text #12

Max Range: 1.1913. Min Range: -1.1913

Max Act: 0.9595. Min Act: -0.0000

Data Index: 948789 (The Pile)

Max Activating Token Index: 479

Text #13

Max Range: 1.1913. Min Range: -1.1913

Max Act: 1.0144. Min Act: -0.0000

Data Index: 732065 (The Pile)

Max Activating Token Index: 179

Text #14

Max Range: 1.1913. Min Range: -1.1913

Max Act: 0.9906. Min Act: -0.0000

Data Index: 1607486 (The Pile)

Max Activating Token Index: 738

Text #15

Max Range: 1.1913. Min Range: -1.1913

Max Act: 0.9916. Min Act: -0.0000

Data Index: 1782800 (The Pile)

Max Activating Token Index: 79

Text #16

Max Range: 1.1913. Min Range: -1.1913

Max Act: 0.9195. Min Act: -0.0000

Data Index: 901975 (The Pile)

Max Activating Token Index: 1007

Text #17

Max Range: 1.1913. Min Range: -1.1913

Max Act: 0.9704. Min Act: -0.0000

Data Index: 1484589 (The Pile)

Max Activating Token Index: 125

Text #18

Max Range: 1.1913. Min Range: -1.1913

Max Act: 0.9576. Min Act: -0.0000

Data Index: 961173 (The Pile)

Max Activating Token Index: 360

Text #19

Max Range: 1.1913. Min Range: -1.1913

Max Act: 0.9622. Min Act: -0.0000

Data Index: 1550597 (The Pile)

Max Activating Token Index: 544