Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4353 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 9.1966. Min Range: -9.1966

Max Act: 9.1966. Min Act: -0.0000

Data Index: 84193 (The Pile)

Max Activating Token Index: 806

Text #1

Max Range: 9.1966. Min Range: -9.1966

Max Act: 9.0946. Min Act: -0.0000

Data Index: 1616527 (The Pile)

Max Activating Token Index: 282

Text #2

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.6947. Min Act: -0.0000

Data Index: 1631497 (The Pile)

Max Activating Token Index: 955

Text #3

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.7752. Min Act: -0.0000

Data Index: 1100683 (The Pile)

Max Activating Token Index: 474

Text #4

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.5544. Min Act: -0.0000

Data Index: 552262 (The Pile)

Max Activating Token Index: 810

Text #5

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.3783. Min Act: -0.0000

Data Index: 393713 (The Pile)

Max Activating Token Index: 960

Text #6

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.4354. Min Act: -0.0000

Data Index: 1108291 (The Pile)

Max Activating Token Index: 803

Text #7

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.1227. Min Act: -0.0000

Data Index: 623843 (The Pile)

Max Activating Token Index: 142

Text #8

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.2704. Min Act: -0.0000

Data Index: 14294 (The Pile)

Max Activating Token Index: 534

Text #9

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.5951. Min Act: -0.0000

Data Index: 1297898 (The Pile)

Max Activating Token Index: 1020

Text #10

Max Range: 9.1966. Min Range: -9.1966

Max Act: 7.7859. Min Act: -0.0000

Data Index: 1713963 (The Pile)

Max Activating Token Index: 1019

Text #11

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.4806. Min Act: -0.0000

Data Index: 1215919 (The Pile)

Max Activating Token Index: 937

Text #12

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.2964. Min Act: -0.0000

Data Index: 104727 (The Pile)

Max Activating Token Index: 250

Text #13

Max Range: 9.1966. Min Range: -9.1966

Max Act: 7.9326. Min Act: -0.0000

Data Index: 121493 (The Pile)

Max Activating Token Index: 689

Text #14

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.2757. Min Act: -0.0000

Data Index: 1832191 (The Pile)

Max Activating Token Index: 327

Text #15

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.1678. Min Act: -0.0000

Data Index: 1411240 (The Pile)

Max Activating Token Index: 270

Text #16

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.5181. Min Act: -0.0000

Data Index: 317237 (The Pile)

Max Activating Token Index: 520

Text #17

Max Range: 9.1966. Min Range: -9.1966

Max Act: 7.5888. Min Act: -0.0000

Data Index: 998010 (The Pile)

Max Activating Token Index: 769

Text #18

Max Range: 9.1966. Min Range: -9.1966

Max Act: 7.8868. Min Act: -0.0000

Data Index: 1599930 (The Pile)

Max Activating Token Index: 618

Text #19

Max Range: 9.1966. Min Range: -9.1966

Max Act: 8.0779. Min Act: -0.0000

Data Index: 1161183 (The Pile)

Max Activating Token Index: 409