Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4140 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.3904. Min Act: -0.0000

Data Index: 937343 (The Pile)

Max Activating Token Index: 221

Text #1

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4046. Min Act: -0.0000

Data Index: 489584 (The Pile)

Max Activating Token Index: 171

Text #2

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4100. Min Act: -0.0000

Data Index: 633582 (The Pile)

Max Activating Token Index: 175

Text #3

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.3941. Min Act: -0.0000

Data Index: 1948438 (The Pile)

Max Activating Token Index: 289

Text #4

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4297. Min Act: -0.0000

Data Index: 935382 (The Pile)

Max Activating Token Index: 176

Text #5

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4030. Min Act: -0.0000

Data Index: 1953034 (The Pile)

Max Activating Token Index: 180

Text #6

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4278. Min Act: -0.0000

Data Index: 1524945 (The Pile)

Max Activating Token Index: 147

Text #7

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4118. Min Act: -0.0000

Data Index: 324770 (The Pile)

Max Activating Token Index: 273

Text #8

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4152. Min Act: -0.0000

Data Index: 1697097 (The Pile)

Max Activating Token Index: 302

Text #9

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4002. Min Act: -0.0000

Data Index: 566130 (The Pile)

Max Activating Token Index: 177

Text #10

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4123. Min Act: -0.0000

Data Index: 1307818 (The Pile)

Max Activating Token Index: 254

Text #11

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4026. Min Act: -0.0000

Data Index: 1295891 (The Pile)

Max Activating Token Index: 150

Text #12

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.3910. Min Act: -0.0000

Data Index: 620663 (The Pile)

Max Activating Token Index: 200

Text #13

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.3708. Min Act: -0.0000

Data Index: 1099688 (The Pile)

Max Activating Token Index: 306

Text #14

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4028. Min Act: -0.0000

Data Index: 140438 (The Pile)

Max Activating Token Index: 196

Text #15

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4154. Min Act: -0.0000

Data Index: 1171777 (The Pile)

Max Activating Token Index: 811

Text #16

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.3864. Min Act: -0.0000

Data Index: 885776 (The Pile)

Max Activating Token Index: 133

Text #17

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.4122. Min Act: -0.0000

Data Index: 1036276 (The Pile)

Max Activating Token Index: 458

Text #18

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.3508. Min Act: -0.0000

Data Index: 1203580 (The Pile)

Max Activating Token Index: 129

Text #19

Max Range: 0.4297. Min Range: -0.4297

Max Act: 0.3794. Min Act: -0.0000

Data Index: 786462 (The Pile)

Max Activating Token Index: 243