Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2492 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.3460. Min Act: -0.0000

Data Index: 1291283 (The Pile)

Max Activating Token Index: 808

Text #1

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.2933. Min Act: -0.0000

Data Index: 361725 (The Pile)

Max Activating Token Index: 326

Text #2

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.2143. Min Act: -0.0000

Data Index: 722023 (The Pile)

Max Activating Token Index: 374

Text #3

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.2104. Min Act: -0.0000

Data Index: 1140063 (The Pile)

Max Activating Token Index: 765

Text #4

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.2114. Min Act: -0.0000

Data Index: 1251531 (The Pile)

Max Activating Token Index: 805

Text #5

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.2030. Min Act: -0.0000

Data Index: 1621718 (The Pile)

Max Activating Token Index: 446

Text #6

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.1843. Min Act: -0.0000

Data Index: 893946 (The Pile)

Max Activating Token Index: 1018

Text #7

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.3015. Min Act: -0.0000

Data Index: 812239 (The Pile)

Max Activating Token Index: 667

Text #8

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.0824. Min Act: -0.0000

Data Index: 245410 (The Pile)

Max Activating Token Index: 718

Text #9

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.1429. Min Act: -0.0000

Data Index: 1800492 (The Pile)

Max Activating Token Index: 520

Text #10

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.1429. Min Act: -0.0000

Data Index: 1195401 (The Pile)

Max Activating Token Index: 129

Text #11

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.0682. Min Act: -0.0000

Data Index: 1141066 (The Pile)

Max Activating Token Index: 189

Text #12

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.1797. Min Act: -0.0000

Data Index: 1902407 (The Pile)

Max Activating Token Index: 390

Text #13

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.0913. Min Act: -0.0000

Data Index: 453533 (The Pile)

Max Activating Token Index: 675

Text #14

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.0887. Min Act: -0.0000

Data Index: 1626342 (The Pile)

Max Activating Token Index: 332

Text #15

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.1326. Min Act: -0.0000

Data Index: 1076062 (The Pile)

Max Activating Token Index: 296

Text #16

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.0263. Min Act: -0.0000

Data Index: 1696191 (The Pile)

Max Activating Token Index: 991

Text #17

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.0078. Min Act: -0.0000

Data Index: 951356 (The Pile)

Max Activating Token Index: 596

Text #18

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.0084. Min Act: -0.0000

Data Index: 866809 (The Pile)

Max Activating Token Index: 321

Text #19

Max Range: 1.3460. Min Range: -1.3460

Max Act: 1.1347. Min Act: -0.0000

Data Index: 1253880 (The Pile)

Max Activating Token Index: 928