Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1382 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.3472. Min Act: -0.0000

Data Index: 106917 (The Pile)

Max Activating Token Index: 445

Text #1

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1961. Min Act: -0.0000

Data Index: 1405908 (The Pile)

Max Activating Token Index: 213

Text #2

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.2094. Min Act: -0.0000

Data Index: 1693991 (The Pile)

Max Activating Token Index: 965

Text #3

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.2036. Min Act: -0.0001

Data Index: 493924 (The Pile)

Max Activating Token Index: 948

Text #4

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.2040. Min Act: -0.0000

Data Index: 405797 (The Pile)

Max Activating Token Index: 1018

Text #5

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1814. Min Act: -0.0000

Data Index: 1516500 (The Pile)

Max Activating Token Index: 1016

Text #6

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.2001. Min Act: -0.0000

Data Index: 1350407 (The Pile)

Max Activating Token Index: 273

Text #7

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1734. Min Act: -0.0000

Data Index: 1246023 (The Pile)

Max Activating Token Index: 839

Text #8

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1852. Min Act: -0.0000

Data Index: 872101 (The Pile)

Max Activating Token Index: 540

Text #9

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1877. Min Act: -0.0000

Data Index: 45179 (The Pile)

Max Activating Token Index: 695

Text #10

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1874. Min Act: -0.0000

Data Index: 1704837 (The Pile)

Max Activating Token Index: 313

Text #11

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1813. Min Act: -0.0000

Data Index: 471637 (The Pile)

Max Activating Token Index: 1020

Text #12

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1743. Min Act: -0.0000

Data Index: 127053 (The Pile)

Max Activating Token Index: 262

Text #13

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1676. Min Act: -0.0000

Data Index: 1813229 (The Pile)

Max Activating Token Index: 730

Text #14

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1772. Min Act: -0.0000

Data Index: 1452898 (The Pile)

Max Activating Token Index: 210

Text #15

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1585. Min Act: -0.0000

Data Index: 201890 (The Pile)

Max Activating Token Index: 459

Text #16

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1525. Min Act: -0.0000

Data Index: 1785738 (The Pile)

Max Activating Token Index: 779

Text #17

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1549. Min Act: -0.0000

Data Index: 1327406 (The Pile)

Max Activating Token Index: 93

Text #18

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1596. Min Act: -0.0000

Data Index: 1219122 (The Pile)

Max Activating Token Index: 87

Text #19

Max Range: 0.3472. Min Range: -0.3472

Max Act: 0.1858. Min Act: -0.0000

Data Index: 1845851 (The Pile)

Max Activating Token Index: 830