Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4874 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.4992. Min Act: -0.0000

Data Index: 836352 (The Pile)

Max Activating Token Index: 900

Text #1

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.9284. Min Act: -0.0000

Data Index: 18383 (The Pile)

Max Activating Token Index: 963

Text #2

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.9483. Min Act: -0.0000

Data Index: 1605539 (The Pile)

Max Activating Token Index: 902

Text #3

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.1427. Min Act: -0.0000

Data Index: 1752720 (The Pile)

Max Activating Token Index: 893

Text #4

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.7478. Min Act: -0.0000

Data Index: 472850 (The Pile)

Max Activating Token Index: 838

Text #5

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.9638. Min Act: -0.0000

Data Index: 1766790 (The Pile)

Max Activating Token Index: 711

Text #6

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.7755. Min Act: -0.0000

Data Index: 807352 (The Pile)

Max Activating Token Index: 561

Text #7

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.6788. Min Act: -0.0000

Data Index: 812881 (The Pile)

Max Activating Token Index: 797

Text #8

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.8883. Min Act: -0.0000

Data Index: 961398 (The Pile)

Max Activating Token Index: 733

Text #9

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.1408. Min Act: -0.0000

Data Index: 1013170 (The Pile)

Max Activating Token Index: 22

Text #10

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.8096. Min Act: -0.0000

Data Index: 1520154 (The Pile)

Max Activating Token Index: 887

Text #11

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.9400. Min Act: -0.0000

Data Index: 1843261 (The Pile)

Max Activating Token Index: 1020

Text #12

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.2407. Min Act: -0.0000

Data Index: 1884807 (The Pile)

Max Activating Token Index: 390

Text #13

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.6744. Min Act: -0.0000

Data Index: 1378399 (The Pile)

Max Activating Token Index: 647

Text #14

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.0881. Min Act: -0.0000

Data Index: 1717017 (The Pile)

Max Activating Token Index: 365

Text #15

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.2209. Min Act: -0.0000

Data Index: 1590986 (The Pile)

Max Activating Token Index: 428

Text #16

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.0933. Min Act: -0.0000

Data Index: 1324938 (The Pile)

Max Activating Token Index: 810

Text #17

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.7751. Min Act: -0.0000

Data Index: 1789968 (The Pile)

Max Activating Token Index: 396

Text #18

Max Range: 3.7755. Min Range: -3.7755

Max Act: 2.7558. Min Act: -0.0000

Data Index: 550475 (The Pile)

Max Activating Token Index: 853

Text #19

Max Range: 3.7755. Min Range: -3.7755

Max Act: 3.2102. Min Act: -0.0000

Data Index: 1103923 (The Pile)

Max Activating Token Index: 472