Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 48 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.2518. Min Act: -0.0000

Data Index: 212639 (The Pile)

Max Activating Token Index: 472

Text #1

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.2289. Min Act: -0.0000

Data Index: 466266 (The Pile)

Max Activating Token Index: 319

Text #2

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1935. Min Act: -0.0000

Data Index: 465425 (The Pile)

Max Activating Token Index: 951

Text #3

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1536. Min Act: -0.0000

Data Index: 416834 (The Pile)

Max Activating Token Index: 847

Text #4

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1693. Min Act: -0.0000

Data Index: 293170 (The Pile)

Max Activating Token Index: 661

Text #5

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1666. Min Act: -0.0000

Data Index: 1387984 (The Pile)

Max Activating Token Index: 913

Text #6

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1618. Min Act: -0.0000

Data Index: 775245 (The Pile)

Max Activating Token Index: 387

Text #7

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1522. Min Act: -0.0000

Data Index: 574085 (The Pile)

Max Activating Token Index: 490

Text #8

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1514. Min Act: -0.0000

Data Index: 17137 (The Pile)

Max Activating Token Index: 540

Text #9

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1597. Min Act: -0.0000

Data Index: 126214 (The Pile)

Max Activating Token Index: 739

Text #10

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1468. Min Act: -0.0000

Data Index: 1315010 (The Pile)

Max Activating Token Index: 407

Text #11

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1373. Min Act: -0.0000

Data Index: 986379 (The Pile)

Max Activating Token Index: 639

Text #12

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1338. Min Act: -0.0000

Data Index: 197037 (The Pile)

Max Activating Token Index: 658

Text #13

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1308. Min Act: -0.0000

Data Index: 191935 (The Pile)

Max Activating Token Index: 1013

Text #14

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1357. Min Act: -0.0000

Data Index: 202359 (The Pile)

Max Activating Token Index: 918

Text #15

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1392. Min Act: -0.0000

Data Index: 1842583 (The Pile)

Max Activating Token Index: 844

Text #16

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1180. Min Act: -0.0000

Data Index: 1765435 (The Pile)

Max Activating Token Index: 873

Text #17

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1221. Min Act: -0.0000

Data Index: 576345 (The Pile)

Max Activating Token Index: 706

Text #18

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1243. Min Act: -0.0000

Data Index: 552273 (The Pile)

Max Activating Token Index: 896

Text #19

Max Range: 0.2518. Min Range: -0.2518

Max Act: 0.1255. Min Act: -0.0000

Data Index: 67240 (The Pile)

Max Activating Token Index: 355