Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1717 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.4483. Min Act: -0.0000

Data Index: 1262977 (The Pile)

Max Activating Token Index: 459

Text #1

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.2357. Min Act: -0.0000

Data Index: 1606932 (The Pile)

Max Activating Token Index: 381

Text #2

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.2785. Min Act: -0.0000

Data Index: 1136952 (The Pile)

Max Activating Token Index: 170

Text #3

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.2638. Min Act: -0.0000

Data Index: 400271 (The Pile)

Max Activating Token Index: 650

Text #4

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.2763. Min Act: -0.0000

Data Index: 1485545 (The Pile)

Max Activating Token Index: 801

Text #5

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.2216. Min Act: -0.0000

Data Index: 350625 (The Pile)

Max Activating Token Index: 432

Text #6

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.1869. Min Act: -0.0000

Data Index: 1853035 (The Pile)

Max Activating Token Index: 754

Text #7

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.2258. Min Act: -0.0000

Data Index: 404047 (The Pile)

Max Activating Token Index: 285

Text #8

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.1959. Min Act: -0.0000

Data Index: 1396660 (The Pile)

Max Activating Token Index: 617

Text #9

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.1807. Min Act: -0.0000

Data Index: 1105833 (The Pile)

Max Activating Token Index: 806

Text #10

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.1211. Min Act: -0.0000

Data Index: 1981605 (The Pile)

Max Activating Token Index: 710

Text #11

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0828. Min Act: -0.0000

Data Index: 1992854 (The Pile)

Max Activating Token Index: 961

Text #12

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0996. Min Act: -0.0000

Data Index: 300378 (The Pile)

Max Activating Token Index: 379

Text #13

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0951. Min Act: -0.0000

Data Index: 1463265 (The Pile)

Max Activating Token Index: 455

Text #14

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0898. Min Act: -0.0000

Data Index: 658339 (The Pile)

Max Activating Token Index: 362

Text #15

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0876. Min Act: -0.0000

Data Index: 1149228 (The Pile)

Max Activating Token Index: 91

Text #16

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0665. Min Act: -0.0000

Data Index: 549033 (The Pile)

Max Activating Token Index: 632

Text #17

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0610. Min Act: -0.0000

Data Index: 1418205 (The Pile)

Max Activating Token Index: 280

Text #18

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.1202. Min Act: -0.0000

Data Index: 1342468 (The Pile)

Max Activating Token Index: 542

Text #19

Max Range: 1.4483. Min Range: -1.4483

Max Act: 1.0531. Min Act: -0.0000

Data Index: 1308624 (The Pile)

Max Activating Token Index: 957