Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1720 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3011. Min Range: -1.3011

Max Act: 1.3011. Min Act: -0.0000

Data Index: 564345 (The Pile)

Max Activating Token Index: 331

Text #1

Max Range: 1.3011. Min Range: -1.3011

Max Act: 1.0034. Min Act: -0.0000

Data Index: 859939 (The Pile)

Max Activating Token Index: 176

Text #2

Max Range: 1.3011. Min Range: -1.3011

Max Act: 1.0486. Min Act: -0.0000

Data Index: 1503758 (The Pile)

Max Activating Token Index: 429

Text #3

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9810. Min Act: -0.0000

Data Index: 746288 (The Pile)

Max Activating Token Index: 417

Text #4

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9876. Min Act: -0.0000

Data Index: 988402 (The Pile)

Max Activating Token Index: 60

Text #5

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9515. Min Act: -0.0000

Data Index: 1534432 (The Pile)

Max Activating Token Index: 264

Text #6

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9193. Min Act: -0.0000

Data Index: 319189 (The Pile)

Max Activating Token Index: 224

Text #7

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9986. Min Act: -0.0000

Data Index: 1839888 (The Pile)

Max Activating Token Index: 100

Text #8

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9675. Min Act: -0.0000

Data Index: 1524489 (The Pile)

Max Activating Token Index: 189

Text #9

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9899. Min Act: -0.0000

Data Index: 1092462 (The Pile)

Max Activating Token Index: 948

Text #10

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9378. Min Act: -0.0000

Data Index: 1588658 (The Pile)

Max Activating Token Index: 711

Text #11

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9531. Min Act: -0.0000

Data Index: 1620338 (The Pile)

Max Activating Token Index: 73

Text #12

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9137. Min Act: -0.0000

Data Index: 900798 (The Pile)

Max Activating Token Index: 252

Text #13

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9342. Min Act: -0.0000

Data Index: 1866104 (The Pile)

Max Activating Token Index: 65

Text #14

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9315. Min Act: -0.0000

Data Index: 251606 (The Pile)

Max Activating Token Index: 919

Text #15

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.8523. Min Act: -0.0000

Data Index: 1881664 (The Pile)

Max Activating Token Index: 745

Text #16

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.8514. Min Act: -0.0000

Data Index: 744701 (The Pile)

Max Activating Token Index: 103

Text #17

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.9251. Min Act: -0.0000

Data Index: 1619846 (The Pile)

Max Activating Token Index: 132

Text #18

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.8593. Min Act: -0.0000

Data Index: 57099 (The Pile)

Max Activating Token Index: 297

Text #19

Max Range: 1.3011. Min Range: -1.3011

Max Act: 0.8865. Min Act: -0.0000

Data Index: 776795 (The Pile)

Max Activating Token Index: 96