Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3588 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1513. Min Range: -3.1513

Max Act: 3.1513. Min Act: -0.0000

Data Index: 768947 (The Pile)

Max Activating Token Index: 394

Text #1

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.7088. Min Act: -0.0000

Data Index: 1220302 (The Pile)

Max Activating Token Index: 955

Text #2

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.7310. Min Act: -0.0000

Data Index: 428867 (The Pile)

Max Activating Token Index: 648

Text #3

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.7484. Min Act: -0.0000

Data Index: 675903 (The Pile)

Max Activating Token Index: 987

Text #4

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.6380. Min Act: -0.0000

Data Index: 309508 (The Pile)

Max Activating Token Index: 984

Text #5

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.6829. Min Act: -0.0000

Data Index: 1888378 (The Pile)

Max Activating Token Index: 358

Text #6

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.5322. Min Act: -0.0000

Data Index: 803225 (The Pile)

Max Activating Token Index: 874

Text #7

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.4675. Min Act: -0.0000

Data Index: 1134443 (The Pile)

Max Activating Token Index: 210

Text #8

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.4781. Min Act: -0.0000

Data Index: 1987755 (The Pile)

Max Activating Token Index: 755

Text #9

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.5500. Min Act: -0.0000

Data Index: 1290772 (The Pile)

Max Activating Token Index: 469

Text #10

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.4365. Min Act: -0.0000

Data Index: 727568 (The Pile)

Max Activating Token Index: 934

Text #11

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.2308. Min Act: -0.0000

Data Index: 873918 (The Pile)

Max Activating Token Index: 1006

Text #12

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.2869. Min Act: -0.0000

Data Index: 1703280 (The Pile)

Max Activating Token Index: 398

Text #13

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.3925. Min Act: -0.0000

Data Index: 1100886 (The Pile)

Max Activating Token Index: 530

Text #14

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.1920. Min Act: -0.0000

Data Index: 1998422 (The Pile)

Max Activating Token Index: 317

Text #15

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.0753. Min Act: -0.0000

Data Index: 842202 (The Pile)

Max Activating Token Index: 562

Text #16

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.5390. Min Act: -0.0000

Data Index: 1174670 (The Pile)

Max Activating Token Index: 1019

Text #17

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.0464. Min Act: -0.0000

Data Index: 503596 (The Pile)

Max Activating Token Index: 387

Text #18

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.2973. Min Act: -0.0000

Data Index: 367424 (The Pile)

Max Activating Token Index: 306

Text #19

Max Range: 3.1513. Min Range: -3.1513

Max Act: 2.1030. Min Act: -0.0000

Data Index: 714073 (The Pile)

Max Activating Token Index: 335