Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2842 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 3.5171. Min Range: -3.5171

Max Act: 3.5171. Min Act: -0.0000

Data Index: 807425 (The Pile)

Max Activating Token Index: 240

Text #1

Max Range: 3.5171. Min Range: -3.5171

Max Act: 3.5046. Min Act: -0.0000

Data Index: 902689 (The Pile)

Max Activating Token Index: 214

Text #2

Max Range: 3.5171. Min Range: -3.5171

Max Act: 3.2818. Min Act: -0.0000

Data Index: 1803933 (The Pile)

Max Activating Token Index: 560

Text #3

Max Range: 3.5171. Min Range: -3.5171

Max Act: 3.0968. Min Act: -0.0000

Data Index: 1991449 (The Pile)

Max Activating Token Index: 397

Text #4

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.7163. Min Act: -0.0000

Data Index: 139899 (The Pile)

Max Activating Token Index: 529

Text #5

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.9427. Min Act: -0.0000

Data Index: 650380 (The Pile)

Max Activating Token Index: 360

Text #6

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.9550. Min Act: -0.0000

Data Index: 1538360 (The Pile)

Max Activating Token Index: 826

Text #7

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.9063. Min Act: -0.0000

Data Index: 1439531 (The Pile)

Max Activating Token Index: 647

Text #8

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.6671. Min Act: -0.0000

Data Index: 1802942 (The Pile)

Max Activating Token Index: 861

Text #9

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.7722. Min Act: -0.0000

Data Index: 1020560 (The Pile)

Max Activating Token Index: 561

Text #10

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.7362. Min Act: -0.0000

Data Index: 1324558 (The Pile)

Max Activating Token Index: 130

Text #11

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.8408. Min Act: -0.0000

Data Index: 1602446 (The Pile)

Max Activating Token Index: 428

Text #12

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.6904. Min Act: -0.0000

Data Index: 845322 (The Pile)

Max Activating Token Index: 932

Text #13

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.2748. Min Act: -0.0000

Data Index: 260904 (The Pile)

Max Activating Token Index: 512

Text #14

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.6404. Min Act: -0.0000

Data Index: 1899703 (The Pile)

Max Activating Token Index: 676

Text #15

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.7534. Min Act: -0.0000

Data Index: 1827522 (The Pile)

Max Activating Token Index: 666

Text #16

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.4453. Min Act: -0.0000

Data Index: 1106588 (The Pile)

Max Activating Token Index: 635

Text #17

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.5872. Min Act: -0.0000

Data Index: 842571 (The Pile)

Max Activating Token Index: 784

Text #18

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.6741. Min Act: -0.0000

Data Index: 634633 (The Pile)

Max Activating Token Index: 550

Text #19

Max Range: 3.5171. Min Range: -3.5171

Max Act: 2.5524. Min Act: -0.0000

Data Index: 1859505 (The Pile)

Max Activating Token Index: 534