Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1977 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.4485. Min Act: -0.0000

Data Index: 1697230 (The Pile)

Max Activating Token Index: 792

Text #1

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.4685. Min Act: -0.0000

Data Index: 1717835 (The Pile)

Max Activating Token Index: 346

Text #2

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.3483. Min Act: -0.0000

Data Index: 528680 (The Pile)

Max Activating Token Index: 924

Text #3

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.3766. Min Act: -0.0000

Data Index: 1819770 (The Pile)

Max Activating Token Index: 303

Text #4

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2972. Min Act: -0.0000

Data Index: 1432185 (The Pile)

Max Activating Token Index: 61

Text #5

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.3642. Min Act: -0.0000

Data Index: 1364872 (The Pile)

Max Activating Token Index: 493

Text #6

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.3351. Min Act: -0.0000

Data Index: 1522675 (The Pile)

Max Activating Token Index: 453

Text #7

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2711. Min Act: -0.0000

Data Index: 982259 (The Pile)

Max Activating Token Index: 201

Text #8

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.3864. Min Act: -0.0000

Data Index: 865520 (The Pile)

Max Activating Token Index: 555

Text #9

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2067. Min Act: -0.0000

Data Index: 1554033 (The Pile)

Max Activating Token Index: 797

Text #10

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2048. Min Act: -0.0000

Data Index: 1841397 (The Pile)

Max Activating Token Index: 660

Text #11

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2422. Min Act: -0.0000

Data Index: 1533747 (The Pile)

Max Activating Token Index: 941

Text #12

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.1513. Min Act: -0.0000

Data Index: 1437015 (The Pile)

Max Activating Token Index: 872

Text #13

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2151. Min Act: -0.0000

Data Index: 1831897 (The Pile)

Max Activating Token Index: 905

Text #14

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2392. Min Act: -0.0000

Data Index: 1944501 (The Pile)

Max Activating Token Index: 43

Text #15

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2329. Min Act: -0.0000

Data Index: 1716193 (The Pile)

Max Activating Token Index: 550

Text #16

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.2605. Min Act: -0.0000

Data Index: 531288 (The Pile)

Max Activating Token Index: 481

Text #17

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.1234. Min Act: -0.0000

Data Index: 1051113 (The Pile)

Max Activating Token Index: 10

Text #18

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.1788. Min Act: -0.0000

Data Index: 1214240 (The Pile)

Max Activating Token Index: 843

Text #19

Max Range: 1.4685. Min Range: -1.4685

Max Act: 1.1329. Min Act: -0.0000

Data Index: 694579 (The Pile)

Max Activating Token Index: 948