Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 637 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.7447. Min Act: -0.0000

Data Index: 1951209 (The Pile)

Max Activating Token Index: 906

Text #1

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.9081. Min Act: -0.0000

Data Index: 57307 (The Pile)

Max Activating Token Index: 461

Text #2

Max Range: 3.1344. Min Range: -3.1344

Max Act: 3.1344. Min Act: -0.0000

Data Index: 810934 (The Pile)

Max Activating Token Index: 572

Text #3

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.3487. Min Act: -0.0000

Data Index: 1394474 (The Pile)

Max Activating Token Index: 133

Text #4

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.0666. Min Act: -0.0000

Data Index: 758966 (The Pile)

Max Activating Token Index: 990

Text #5

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.3892. Min Act: -0.0000

Data Index: 239111 (The Pile)

Max Activating Token Index: 272

Text #6

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.3661. Min Act: -0.0000

Data Index: 1689403 (The Pile)

Max Activating Token Index: 856

Text #7

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.3396. Min Act: -0.0000

Data Index: 1641341 (The Pile)

Max Activating Token Index: 734

Text #8

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.6053. Min Act: -0.0000

Data Index: 123883 (The Pile)

Max Activating Token Index: 199

Text #9

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.4484. Min Act: -0.0000

Data Index: 1636126 (The Pile)

Max Activating Token Index: 860

Text #10

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.3053. Min Act: -0.0000

Data Index: 227140 (The Pile)

Max Activating Token Index: 717

Text #11

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.2537. Min Act: -0.0000

Data Index: 1393149 (The Pile)

Max Activating Token Index: 635

Text #12

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.2247. Min Act: -0.0000

Data Index: 558650 (The Pile)

Max Activating Token Index: 860

Text #13

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.3249. Min Act: -0.0000

Data Index: 588842 (The Pile)

Max Activating Token Index: 981

Text #14

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.2297. Min Act: -0.0000

Data Index: 142405 (The Pile)

Max Activating Token Index: 509

Text #15

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.3290. Min Act: -0.0000

Data Index: 39282 (The Pile)

Max Activating Token Index: 289

Text #16

Max Range: 3.1344. Min Range: -3.1344

Max Act: 1.9605. Min Act: -0.0000

Data Index: 181031 (The Pile)

Max Activating Token Index: 60

Text #17

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.0238. Min Act: -0.0000

Data Index: 313545 (The Pile)

Max Activating Token Index: 686

Text #18

Max Range: 3.1344. Min Range: -3.1344

Max Act: 2.2355. Min Act: -0.0000

Data Index: 784910 (The Pile)

Max Activating Token Index: 315

Text #19

Max Range: 3.1344. Min Range: -3.1344

Max Act: 1.7062. Min Act: -0.0000

Data Index: 1121692 (The Pile)

Max Activating Token Index: 750