Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2038 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.4785. Min Act: -0.0000

Data Index: 925747 (The Pile)

Max Activating Token Index: 342

Text #1

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.4378. Min Act: -0.0000

Data Index: 131128 (The Pile)

Max Activating Token Index: 978

Text #2

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.4212. Min Act: -0.0000

Data Index: 1669188 (The Pile)

Max Activating Token Index: 693

Text #3

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.4234. Min Act: -0.0000

Data Index: 1691410 (The Pile)

Max Activating Token Index: 821

Text #4

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.4005. Min Act: -0.0000

Data Index: 1456280 (The Pile)

Max Activating Token Index: 1014

Text #5

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3588. Min Act: -0.0000

Data Index: 1293894 (The Pile)

Max Activating Token Index: 627

Text #6

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3709. Min Act: -0.0000

Data Index: 345 (The Pile)

Max Activating Token Index: 1012

Text #7

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3793. Min Act: -0.0000

Data Index: 870739 (The Pile)

Max Activating Token Index: 440

Text #8

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3452. Min Act: -0.0000

Data Index: 364082 (The Pile)

Max Activating Token Index: 965

Text #9

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3276. Min Act: -0.0000

Data Index: 1987581 (The Pile)

Max Activating Token Index: 147

Text #10

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3685. Min Act: -0.0000

Data Index: 728548 (The Pile)

Max Activating Token Index: 406

Text #11

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3666. Min Act: -0.0000

Data Index: 596265 (The Pile)

Max Activating Token Index: 143

Text #12

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3093. Min Act: -0.0000

Data Index: 1884897 (The Pile)

Max Activating Token Index: 992

Text #13

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3681. Min Act: -0.0000

Data Index: 1742947 (The Pile)

Max Activating Token Index: 299

Text #14

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3579. Min Act: -0.0000

Data Index: 169749 (The Pile)

Max Activating Token Index: 892

Text #15

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3292. Min Act: -0.0000

Data Index: 1646706 (The Pile)

Max Activating Token Index: 317

Text #16

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.2857. Min Act: -0.0000

Data Index: 1909994 (The Pile)

Max Activating Token Index: 416

Text #17

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.2955. Min Act: -0.0000

Data Index: 1913967 (The Pile)

Max Activating Token Index: 876

Text #18

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.2917. Min Act: -0.0000

Data Index: 1187224 (The Pile)

Max Activating Token Index: 312

Text #19

Max Range: 0.4785. Min Range: -0.4785

Max Act: 0.3046. Min Act: -0.0000

Data Index: 415814 (The Pile)

Max Activating Token Index: 171