Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1579 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.4733. Min Act: -0.0000

Data Index: 200643 (The Pile)

Max Activating Token Index: 739

Text #1

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.4635. Min Act: -0.0000

Data Index: 364335 (The Pile)

Max Activating Token Index: 557

Text #2

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.4439. Min Act: -0.0000

Data Index: 1715794 (The Pile)

Max Activating Token Index: 471

Text #3

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.4380. Min Act: -0.0000

Data Index: 22634 (The Pile)

Max Activating Token Index: 760

Text #4

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.4179. Min Act: -0.0000

Data Index: 235702 (The Pile)

Max Activating Token Index: 35

Text #5

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3797. Min Act: -0.0000

Data Index: 1491165 (The Pile)

Max Activating Token Index: 704

Text #6

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.4155. Min Act: -0.0000

Data Index: 532239 (The Pile)

Max Activating Token Index: 661

Text #7

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3266. Min Act: -0.0000

Data Index: 356573 (The Pile)

Max Activating Token Index: 203

Text #8

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3548. Min Act: -0.0000

Data Index: 537292 (The Pile)

Max Activating Token Index: 981

Text #9

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3540. Min Act: -0.0000

Data Index: 299265 (The Pile)

Max Activating Token Index: 374

Text #10

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3420. Min Act: -0.0000

Data Index: 414347 (The Pile)

Max Activating Token Index: 825

Text #11

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3898. Min Act: -0.0000

Data Index: 1821874 (The Pile)

Max Activating Token Index: 482

Text #12

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3286. Min Act: -0.0000

Data Index: 875201 (The Pile)

Max Activating Token Index: 119

Text #13

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3388. Min Act: -0.0000

Data Index: 641042 (The Pile)

Max Activating Token Index: 945

Text #14

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3271. Min Act: -0.0000

Data Index: 1274395 (The Pile)

Max Activating Token Index: 182

Text #15

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3135. Min Act: -0.0000

Data Index: 500379 (The Pile)

Max Activating Token Index: 665

Text #16

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.2850. Min Act: -0.0000

Data Index: 63682 (The Pile)

Max Activating Token Index: 140

Text #17

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3206. Min Act: -0.0000

Data Index: 57891 (The Pile)

Max Activating Token Index: 45

Text #18

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3107. Min Act: -0.0000

Data Index: 917832 (The Pile)

Max Activating Token Index: 875

Text #19

Max Range: 0.4733. Min Range: -0.4733

Max Act: 0.3200. Min Act: -0.0000

Data Index: 137531 (The Pile)

Max Activating Token Index: 477