Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 323 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.3695. Min Act: -0.0000

Data Index: 195455 (The Pile)

Max Activating Token Index: 261

Text #1

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.3540. Min Act: -0.0000

Data Index: 920746 (The Pile)

Max Activating Token Index: 483

Text #2

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.3041. Min Act: -0.0000

Data Index: 428868 (The Pile)

Max Activating Token Index: 803

Text #3

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2870. Min Act: -0.0000

Data Index: 696675 (The Pile)

Max Activating Token Index: 394

Text #4

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2918. Min Act: -0.0000

Data Index: 282351 (The Pile)

Max Activating Token Index: 304

Text #5

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2761. Min Act: -0.0000

Data Index: 1150841 (The Pile)

Max Activating Token Index: 219

Text #6

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2591. Min Act: -0.0000

Data Index: 972961 (The Pile)

Max Activating Token Index: 692

Text #7

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2251. Min Act: -0.0000

Data Index: 213049 (The Pile)

Max Activating Token Index: 935

Text #8

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2560. Min Act: -0.0000

Data Index: 1091822 (The Pile)

Max Activating Token Index: 493

Text #9

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2688. Min Act: -0.0000

Data Index: 665718 (The Pile)

Max Activating Token Index: 167

Text #10

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2485. Min Act: -0.0000

Data Index: 1789867 (The Pile)

Max Activating Token Index: 238

Text #11

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2415. Min Act: -0.0000

Data Index: 907634 (The Pile)

Max Activating Token Index: 146

Text #12

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2580. Min Act: -0.0000

Data Index: 183572 (The Pile)

Max Activating Token Index: 784

Text #13

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2434. Min Act: -0.0000

Data Index: 476465 (The Pile)

Max Activating Token Index: 448

Text #14

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2377. Min Act: -0.0000

Data Index: 379963 (The Pile)

Max Activating Token Index: 253

Text #15

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2523. Min Act: -0.0000

Data Index: 932313 (The Pile)

Max Activating Token Index: 925

Text #16

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2194. Min Act: -0.0000

Data Index: 1940258 (The Pile)

Max Activating Token Index: 813

Text #17

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2262. Min Act: -0.0000

Data Index: 1698481 (The Pile)

Max Activating Token Index: 370

Text #18

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2335. Min Act: -0.0000

Data Index: 1580157 (The Pile)

Max Activating Token Index: 965

Text #19

Max Range: 0.3695. Min Range: -0.3695

Max Act: 0.2310. Min Act: -0.0000

Data Index: 52465 (The Pile)

Max Activating Token Index: 444