Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4240 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.5148. Min Act: -0.0000

Data Index: 74558 (The Pile)

Max Activating Token Index: 345

Text #1

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.4480. Min Act: -0.0000

Data Index: 1326392 (The Pile)

Max Activating Token Index: 171

Text #2

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.3818. Min Act: -0.0000

Data Index: 88354 (The Pile)

Max Activating Token Index: 556

Text #3

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.3221. Min Act: -0.0000

Data Index: 45428 (The Pile)

Max Activating Token Index: 204

Text #4

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2422. Min Act: -0.0000

Data Index: 1448112 (The Pile)

Max Activating Token Index: 512

Text #5

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2306. Min Act: -0.0000

Data Index: 797292 (The Pile)

Max Activating Token Index: 901

Text #6

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2564. Min Act: -0.0000

Data Index: 1844467 (The Pile)

Max Activating Token Index: 995

Text #7

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2810. Min Act: -0.0000

Data Index: 1678345 (The Pile)

Max Activating Token Index: 131

Text #8

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2742. Min Act: -0.0000

Data Index: 906178 (The Pile)

Max Activating Token Index: 184

Text #9

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.0866. Min Act: -0.0000

Data Index: 1544181 (The Pile)

Max Activating Token Index: 585

Text #10

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2322. Min Act: -0.0000

Data Index: 691528 (The Pile)

Max Activating Token Index: 567

Text #11

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2379. Min Act: -0.0000

Data Index: 727223 (The Pile)

Max Activating Token Index: 734

Text #12

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.1729. Min Act: -0.0000

Data Index: 213594 (The Pile)

Max Activating Token Index: 194

Text #13

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2289. Min Act: -0.0000

Data Index: 1702065 (The Pile)

Max Activating Token Index: 684

Text #14

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2249. Min Act: -0.0000

Data Index: 1335004 (The Pile)

Max Activating Token Index: 356

Text #15

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.0898. Min Act: -0.0000

Data Index: 1957268 (The Pile)

Max Activating Token Index: 123

Text #16

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.0944. Min Act: -0.0000

Data Index: 202130 (The Pile)

Max Activating Token Index: 147

Text #17

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.1863. Min Act: -0.0000

Data Index: 812968 (The Pile)

Max Activating Token Index: 238

Text #18

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.1825. Min Act: -0.0000

Data Index: 307125 (The Pile)

Max Activating Token Index: 79

Text #19

Max Range: 1.5148. Min Range: -1.5148

Max Act: 1.2022. Min Act: -0.0000

Data Index: 1501143 (The Pile)

Max Activating Token Index: 589