Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1029 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.0319. Min Range: -1.0319

Max Act: 1.0319. Min Act: -0.0000

Data Index: 720941 (The Pile)

Max Activating Token Index: 340

Text #1

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.6687. Min Act: -0.0000

Data Index: 654260 (The Pile)

Max Activating Token Index: 791

Text #2

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.5887. Min Act: -0.0000

Data Index: 477436 (The Pile)

Max Activating Token Index: 813

Text #3

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.6665. Min Act: -0.0000

Data Index: 1769306 (The Pile)

Max Activating Token Index: 545

Text #4

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.5973. Min Act: -0.0000

Data Index: 1124105 (The Pile)

Max Activating Token Index: 856

Text #5

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4985. Min Act: -0.0000

Data Index: 446434 (The Pile)

Max Activating Token Index: 451

Text #6

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4195. Min Act: -0.0000

Data Index: 1354312 (The Pile)

Max Activating Token Index: 651

Text #7

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4204. Min Act: -0.0000

Data Index: 193791 (The Pile)

Max Activating Token Index: 728

Text #8

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4257. Min Act: -0.0000

Data Index: 1977987 (The Pile)

Max Activating Token Index: 1010

Text #9

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4472. Min Act: -0.0000

Data Index: 235022 (The Pile)

Max Activating Token Index: 363

Text #10

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4836. Min Act: -0.0000

Data Index: 1224272 (The Pile)

Max Activating Token Index: 746

Text #11

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4573. Min Act: -0.0000

Data Index: 63750 (The Pile)

Max Activating Token Index: 928

Text #12

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4588. Min Act: -0.0000

Data Index: 1577331 (The Pile)

Max Activating Token Index: 843

Text #13

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.3691. Min Act: -0.0000

Data Index: 308448 (The Pile)

Max Activating Token Index: 202

Text #14

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4274. Min Act: -0.0000

Data Index: 1896596 (The Pile)

Max Activating Token Index: 425

Text #15

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.3760. Min Act: -0.0000

Data Index: 177357 (The Pile)

Max Activating Token Index: 841

Text #16

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.3357. Min Act: -0.0000

Data Index: 765929 (The Pile)

Max Activating Token Index: 184

Text #17

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4172. Min Act: -0.0000

Data Index: 943284 (The Pile)

Max Activating Token Index: 953

Text #18

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.4241. Min Act: -0.0000

Data Index: 774289 (The Pile)

Max Activating Token Index: 380

Text #19

Max Range: 1.0319. Min Range: -1.0319

Max Act: 0.3443. Min Act: -0.0000

Data Index: 864931 (The Pile)

Max Activating Token Index: 923