Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3583 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.8635. Min Act: -0.0000

Data Index: 497319 (The Pile)

Max Activating Token Index: 304

Text #1

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.3758. Min Act: -0.0000

Data Index: 799245 (The Pile)

Max Activating Token Index: 561

Text #2

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.8441. Min Act: -0.0000

Data Index: 1028054 (The Pile)

Max Activating Token Index: 535

Text #3

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.6318. Min Act: -0.0000

Data Index: 103994 (The Pile)

Max Activating Token Index: 15

Text #4

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.5602. Min Act: -0.0000

Data Index: 1870311 (The Pile)

Max Activating Token Index: 376

Text #5

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.6984. Min Act: -0.0000

Data Index: 801292 (The Pile)

Max Activating Token Index: 157

Text #6

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.6855. Min Act: -0.0000

Data Index: 1990713 (The Pile)

Max Activating Token Index: 824

Text #7

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.5807. Min Act: -0.0000

Data Index: 1251241 (The Pile)

Max Activating Token Index: 230

Text #8

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.6907. Min Act: -0.0000

Data Index: 663761 (The Pile)

Max Activating Token Index: 554

Text #9

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.5149. Min Act: -0.0000

Data Index: 661708 (The Pile)

Max Activating Token Index: 749

Text #10

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.2490. Min Act: -0.0000

Data Index: 1346634 (The Pile)

Max Activating Token Index: 714

Text #11

Max Range: 3.8635. Min Range: -3.8635

Max Act: 1.9915. Min Act: -0.0000

Data Index: 1571202 (The Pile)

Max Activating Token Index: 517

Text #12

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.5558. Min Act: -0.0000

Data Index: 849704 (The Pile)

Max Activating Token Index: 155

Text #13

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.4181. Min Act: -0.0000

Data Index: 1521165 (The Pile)

Max Activating Token Index: 547

Text #14

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.3912. Min Act: -0.0000

Data Index: 523536 (The Pile)

Max Activating Token Index: 997

Text #15

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.4089. Min Act: -0.0000

Data Index: 1595286 (The Pile)

Max Activating Token Index: 611

Text #16

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.1205. Min Act: -0.0000

Data Index: 549770 (The Pile)

Max Activating Token Index: 751

Text #17

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.4920. Min Act: -0.0000

Data Index: 416783 (The Pile)

Max Activating Token Index: 561

Text #18

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.6960. Min Act: -0.0000

Data Index: 1447748 (The Pile)

Max Activating Token Index: 160

Text #19

Max Range: 3.8635. Min Range: -3.8635

Max Act: 2.4284. Min Act: -0.0000

Data Index: 672965 (The Pile)

Max Activating Token Index: 999