Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3364 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0366. Min Range: -1.0366

Max Act: 1.0366. Min Act: -0.0000

Data Index: 661560 (The Pile)

Max Activating Token Index: 99

Text #1

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8635. Min Act: -0.0000

Data Index: 1466425 (The Pile)

Max Activating Token Index: 187

Text #2

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8774. Min Act: -0.0000

Data Index: 391149 (The Pile)

Max Activating Token Index: 982

Text #3

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8510. Min Act: -0.0000

Data Index: 1532355 (The Pile)

Max Activating Token Index: 161

Text #4

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8344. Min Act: -0.0000

Data Index: 929692 (The Pile)

Max Activating Token Index: 119

Text #5

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8252. Min Act: -0.0000

Data Index: 1808369 (The Pile)

Max Activating Token Index: 70

Text #6

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8716. Min Act: -0.0000

Data Index: 456902 (The Pile)

Max Activating Token Index: 904

Text #7

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8400. Min Act: -0.0000

Data Index: 182145 (The Pile)

Max Activating Token Index: 321

Text #8

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8020. Min Act: -0.0000

Data Index: 1301400 (The Pile)

Max Activating Token Index: 607

Text #9

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8471. Min Act: -0.0000

Data Index: 101551 (The Pile)

Max Activating Token Index: 326

Text #10

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.7896. Min Act: -0.0000

Data Index: 6510 (The Pile)

Max Activating Token Index: 714

Text #11

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.7952. Min Act: -0.0000

Data Index: 1940530 (The Pile)

Max Activating Token Index: 377

Text #12

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8221. Min Act: -0.0000

Data Index: 1195094 (The Pile)

Max Activating Token Index: 884

Text #13

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8182. Min Act: -0.0000

Data Index: 1542563 (The Pile)

Max Activating Token Index: 457

Text #14

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8139. Min Act: -0.0000

Data Index: 187189 (The Pile)

Max Activating Token Index: 110

Text #15

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.7614. Min Act: -0.0000

Data Index: 1273637 (The Pile)

Max Activating Token Index: 593

Text #16

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.7802. Min Act: -0.0000

Data Index: 237875 (The Pile)

Max Activating Token Index: 702

Text #17

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.8281. Min Act: -0.0000

Data Index: 671129 (The Pile)

Max Activating Token Index: 983

Text #18

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.7389. Min Act: -0.0000

Data Index: 1809446 (The Pile)

Max Activating Token Index: 777

Text #19

Max Range: 1.0366. Min Range: -1.0366

Max Act: 0.7393. Min Act: -0.0000

Data Index: 1091945 (The Pile)

Max Activating Token Index: 30