Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4241 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.8958. Min Act: -0.0000

Data Index: 314859 (The Pile)

Max Activating Token Index: 730

Text #1

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.3382. Min Act: -0.0000

Data Index: 1034533 (The Pile)

Max Activating Token Index: 163

Text #2

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.1557. Min Act: 0.0000

Data Index: 118148 (The Pile)

Max Activating Token Index: 358

Text #3

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.1430. Min Act: -0.0000

Data Index: 524635 (The Pile)

Max Activating Token Index: 352

Text #4

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.1238. Min Act: 0.0000

Data Index: 1220051 (The Pile)

Max Activating Token Index: 103

Text #5

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.0880. Min Act: -0.0000

Data Index: 630265 (The Pile)

Max Activating Token Index: 572

Text #6

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.0603. Min Act: -0.0000

Data Index: 670284 (The Pile)

Max Activating Token Index: 571

Text #7

Max Range: 1.8958. Min Range: -1.8958

Max Act: 1.0430. Min Act: -0.0000

Data Index: 835247 (The Pile)

Max Activating Token Index: 526

Text #8

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.9915. Min Act: -0.0000

Data Index: 935930 (The Pile)

Max Activating Token Index: 54

Text #9

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.9868. Min Act: -0.0000

Data Index: 1629597 (The Pile)

Max Activating Token Index: 224

Text #10

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.8963. Min Act: 0.0000

Data Index: 938384 (The Pile)

Max Activating Token Index: 469

Text #11

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.9237. Min Act: 0.0000

Data Index: 77794 (The Pile)

Max Activating Token Index: 440

Text #12

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.9069. Min Act: 0.0000

Data Index: 239918 (The Pile)

Max Activating Token Index: 513

Text #13

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.9458. Min Act: -0.0000

Data Index: 336464 (The Pile)

Max Activating Token Index: 572

Text #14

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.8768. Min Act: 0.0000

Data Index: 1344462 (The Pile)

Max Activating Token Index: 351

Text #15

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.8703. Min Act: 0.0000

Data Index: 663624 (The Pile)

Max Activating Token Index: 1010

Text #16

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.8056. Min Act: -0.0000

Data Index: 278978 (The Pile)

Max Activating Token Index: 510

Text #17

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.8326. Min Act: -0.0000

Data Index: 1908992 (The Pile)

Max Activating Token Index: 128

Text #18

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.8274. Min Act: -0.0000

Data Index: 1773372 (The Pile)

Max Activating Token Index: 437

Text #19

Max Range: 1.8958. Min Range: -1.8958

Max Act: 0.7702. Min Act: -0.0000

Data Index: 1056342 (The Pile)

Max Activating Token Index: 122