Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3463 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.6008. Min Act: -0.0000

Data Index: 340563 (The Pile)

Max Activating Token Index: 117

Text #1

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.7266. Min Act: -0.0000

Data Index: 1723510 (The Pile)

Max Activating Token Index: 926

Text #2

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.6291. Min Act: -0.0000

Data Index: 1101751 (The Pile)

Max Activating Token Index: 698

Text #3

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.4837. Min Act: -0.0000

Data Index: 406401 (The Pile)

Max Activating Token Index: 324

Text #4

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5524. Min Act: -0.0000

Data Index: 1313110 (The Pile)

Max Activating Token Index: 38

Text #5

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.4924. Min Act: -0.0000

Data Index: 977471 (The Pile)

Max Activating Token Index: 227

Text #6

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5896. Min Act: -0.0000

Data Index: 347514 (The Pile)

Max Activating Token Index: 596

Text #7

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5535. Min Act: -0.0000

Data Index: 600810 (The Pile)

Max Activating Token Index: 194

Text #8

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5930. Min Act: -0.0000

Data Index: 1261472 (The Pile)

Max Activating Token Index: 597

Text #9

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5289. Min Act: -0.0000

Data Index: 1001066 (The Pile)

Max Activating Token Index: 161

Text #10

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5705. Min Act: -0.0000

Data Index: 1111470 (The Pile)

Max Activating Token Index: 863

Text #11

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5689. Min Act: -0.0000

Data Index: 417035 (The Pile)

Max Activating Token Index: 18

Text #12

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.4114. Min Act: -0.0000

Data Index: 896068 (The Pile)

Max Activating Token Index: 137

Text #13

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.4860. Min Act: -0.0000

Data Index: 1612053 (The Pile)

Max Activating Token Index: 60

Text #14

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.4414. Min Act: -0.0000

Data Index: 188999 (The Pile)

Max Activating Token Index: 46

Text #15

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5073. Min Act: -0.0000

Data Index: 1371914 (The Pile)

Max Activating Token Index: 730

Text #16

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.4844. Min Act: -0.0000

Data Index: 1151981 (The Pile)

Max Activating Token Index: 628

Text #17

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5188. Min Act: -0.0000

Data Index: 279257 (The Pile)

Max Activating Token Index: 739

Text #18

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.4654. Min Act: -0.0000

Data Index: 1789098 (The Pile)

Max Activating Token Index: 367

Text #19

Max Range: 0.7266. Min Range: -0.7266

Max Act: 0.5074. Min Act: -0.0000

Data Index: 265666 (The Pile)

Max Activating Token Index: 321