Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 4079 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.4682. Min Act: -0.0000

Data Index: 846089 (The Pile)

Max Activating Token Index: 329

Text #1

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.6566. Min Act: -0.0000

Data Index: 1256993 (The Pile)

Max Activating Token Index: 520

Text #2

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.3741. Min Act: -0.0000

Data Index: 210199 (The Pile)

Max Activating Token Index: 464

Text #3

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.3000. Min Act: -0.0000

Data Index: 370014 (The Pile)

Max Activating Token Index: 841

Text #4

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2678. Min Act: -0.0000

Data Index: 632185 (The Pile)

Max Activating Token Index: 518

Text #5

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.3089. Min Act: -0.0000

Data Index: 950427 (The Pile)

Max Activating Token Index: 528

Text #6

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.3656. Min Act: -0.0000

Data Index: 1400133 (The Pile)

Max Activating Token Index: 606

Text #7

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2341. Min Act: -0.0001

Data Index: 480797 (The Pile)

Max Activating Token Index: 641

Text #8

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.3928. Min Act: -0.0000

Data Index: 52066 (The Pile)

Max Activating Token Index: 105

Text #9

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.3557. Min Act: -0.0000

Data Index: 1481226 (The Pile)

Max Activating Token Index: 733

Text #10

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2128. Min Act: -0.0000

Data Index: 729579 (The Pile)

Max Activating Token Index: 731

Text #11

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2621. Min Act: -0.0000

Data Index: 1964986 (The Pile)

Max Activating Token Index: 476

Text #12

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2520. Min Act: -0.0000

Data Index: 170645 (The Pile)

Max Activating Token Index: 927

Text #13

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2460. Min Act: -0.0000

Data Index: 1690464 (The Pile)

Max Activating Token Index: 128

Text #14

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2681. Min Act: -0.0000

Data Index: 1343868 (The Pile)

Max Activating Token Index: 505

Text #15

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.3804. Min Act: -0.0000

Data Index: 1942091 (The Pile)

Max Activating Token Index: 253

Text #16

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.1776. Min Act: -0.0000

Data Index: 746283 (The Pile)

Max Activating Token Index: 189

Text #17

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.1714. Min Act: -0.0000

Data Index: 1797874 (The Pile)

Max Activating Token Index: 657

Text #18

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2763. Min Act: -0.0000

Data Index: 481766 (The Pile)

Max Activating Token Index: 719

Text #19

Max Range: 1.6566. Min Range: -1.6566

Max Act: 1.2750. Min Act: -0.0000

Data Index: 589026 (The Pile)

Max Activating Token Index: 286