Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 353 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2532. Min Range: -1.2532

Max Act: 1.2532. Min Act: -0.0000

Data Index: 761665 (The Pile)

Max Activating Token Index: 495

Text #1

Max Range: 1.2532. Min Range: -1.2532

Max Act: 1.0628. Min Act: -0.0000

Data Index: 30424 (The Pile)

Max Activating Token Index: 1012

Text #2

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.9472. Min Act: -0.0000

Data Index: 471609 (The Pile)

Max Activating Token Index: 966

Text #3

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.9239. Min Act: -0.0000

Data Index: 1653879 (The Pile)

Max Activating Token Index: 985

Text #4

Max Range: 1.2532. Min Range: -1.2532

Max Act: 1.0344. Min Act: -0.0000

Data Index: 1982441 (The Pile)

Max Activating Token Index: 753

Text #5

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.9468. Min Act: -0.0000

Data Index: 1684445 (The Pile)

Max Activating Token Index: 59

Text #6

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8681. Min Act: -0.0000

Data Index: 1247127 (The Pile)

Max Activating Token Index: 540

Text #7

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8548. Min Act: -0.0000

Data Index: 1718850 (The Pile)

Max Activating Token Index: 645

Text #8

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8626. Min Act: -0.0000

Data Index: 1941454 (The Pile)

Max Activating Token Index: 331

Text #9

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8526. Min Act: -0.0000

Data Index: 1856183 (The Pile)

Max Activating Token Index: 335

Text #10

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8449. Min Act: -0.0000

Data Index: 664874 (The Pile)

Max Activating Token Index: 1018

Text #11

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.7569. Min Act: -0.0000

Data Index: 1466586 (The Pile)

Max Activating Token Index: 288

Text #12

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8230. Min Act: 0.0001

Data Index: 1617735 (The Pile)

Max Activating Token Index: 132

Text #13

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8018. Min Act: -0.0000

Data Index: 407467 (The Pile)

Max Activating Token Index: 623

Text #14

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8058. Min Act: -0.0000

Data Index: 1589550 (The Pile)

Max Activating Token Index: 319

Text #15

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.7578. Min Act: -0.0000

Data Index: 853474 (The Pile)

Max Activating Token Index: 481

Text #16

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8092. Min Act: -0.0000

Data Index: 677421 (The Pile)

Max Activating Token Index: 494

Text #17

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.8301. Min Act: -0.0000

Data Index: 732626 (The Pile)

Max Activating Token Index: 280

Text #18

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.7474. Min Act: -0.0000

Data Index: 751384 (The Pile)

Max Activating Token Index: 616

Text #19

Max Range: 1.2532. Min Range: -1.2532

Max Act: 0.7809. Min Act: -0.0000

Data Index: 119671 (The Pile)

Max Activating Token Index: 567