Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2980 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3877. Min Act: -0.0000

Data Index: 1626555 (The Pile)

Max Activating Token Index: 66

Text #1

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.4289. Min Act: -0.0000

Data Index: 950999 (The Pile)

Max Activating Token Index: 231

Text #2

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.4005. Min Act: -0.0000

Data Index: 128843 (The Pile)

Max Activating Token Index: 533

Text #3

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.4054. Min Act: -0.0000

Data Index: 475863 (The Pile)

Max Activating Token Index: 910

Text #4

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3582. Min Act: -0.0000

Data Index: 1565035 (The Pile)

Max Activating Token Index: 726

Text #5

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3391. Min Act: -0.0000

Data Index: 530971 (The Pile)

Max Activating Token Index: 278

Text #6

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.2847. Min Act: -0.0000

Data Index: 918557 (The Pile)

Max Activating Token Index: 275

Text #7

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3003. Min Act: -0.0000

Data Index: 938178 (The Pile)

Max Activating Token Index: 88

Text #8

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.2741. Min Act: -0.0000

Data Index: 258315 (The Pile)

Max Activating Token Index: 987

Text #9

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3332. Min Act: -0.0000

Data Index: 336182 (The Pile)

Max Activating Token Index: 196

Text #10

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3093. Min Act: -0.0000

Data Index: 1095059 (The Pile)

Max Activating Token Index: 842

Text #11

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3280. Min Act: -0.0000

Data Index: 436545 (The Pile)

Max Activating Token Index: 515

Text #12

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3001. Min Act: -0.0000

Data Index: 203179 (The Pile)

Max Activating Token Index: 36

Text #13

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3026. Min Act: -0.0000

Data Index: 640164 (The Pile)

Max Activating Token Index: 453

Text #14

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3014. Min Act: -0.0000

Data Index: 57472 (The Pile)

Max Activating Token Index: 447

Text #15

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3016. Min Act: -0.0000

Data Index: 366868 (The Pile)

Max Activating Token Index: 74

Text #16

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.2395. Min Act: -0.0000

Data Index: 1741959 (The Pile)

Max Activating Token Index: 744

Text #17

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3017. Min Act: -0.0000

Data Index: 1364406 (The Pile)

Max Activating Token Index: 332

Text #18

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.3054. Min Act: -0.0000

Data Index: 256022 (The Pile)

Max Activating Token Index: 87

Text #19

Max Range: 0.4289. Min Range: -0.4289

Max Act: 0.2882. Min Act: -0.0000

Data Index: 1797788 (The Pile)

Max Activating Token Index: 873