Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 171 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.7785. Min Act: -0.0000

Data Index: 308279 (The Pile)

Max Activating Token Index: 55

Text #1

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.7566. Min Act: -0.0000

Data Index: 1196399 (The Pile)

Max Activating Token Index: 517

Text #2

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.8373. Min Act: -0.0000

Data Index: 1964596 (The Pile)

Max Activating Token Index: 340

Text #3

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.8088. Min Act: -0.0000

Data Index: 138089 (The Pile)

Max Activating Token Index: 436

Text #4

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.7881. Min Act: -0.0000

Data Index: 1517498 (The Pile)

Max Activating Token Index: 371

Text #5

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.7197. Min Act: -0.0000

Data Index: 1877713 (The Pile)

Max Activating Token Index: 73

Text #6

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.7912. Min Act: -0.0000

Data Index: 983242 (The Pile)

Max Activating Token Index: 560

Text #7

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.5983. Min Act: -0.0000

Data Index: 1279321 (The Pile)

Max Activating Token Index: 22

Text #8

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.6798. Min Act: -0.0000

Data Index: 1913926 (The Pile)

Max Activating Token Index: 732

Text #9

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.5844. Min Act: -0.0000

Data Index: 986193 (The Pile)

Max Activating Token Index: 210

Text #10

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.6647. Min Act: -0.0000

Data Index: 264815 (The Pile)

Max Activating Token Index: 331

Text #11

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.7525. Min Act: -0.0000

Data Index: 1399649 (The Pile)

Max Activating Token Index: 605

Text #12

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.6549. Min Act: -0.0000

Data Index: 1271420 (The Pile)

Max Activating Token Index: 316

Text #13

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.5564. Min Act: -0.0000

Data Index: 897282 (The Pile)

Max Activating Token Index: 857

Text #14

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.6228. Min Act: -0.0000

Data Index: 1901416 (The Pile)

Max Activating Token Index: 614

Text #15

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.5511. Min Act: -0.0000

Data Index: 463840 (The Pile)

Max Activating Token Index: 63

Text #16

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.5331. Min Act: -0.0000

Data Index: 1939245 (The Pile)

Max Activating Token Index: 298

Text #17

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.6180. Min Act: -0.0000

Data Index: 498863 (The Pile)

Max Activating Token Index: 769

Text #18

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.5984. Min Act: -0.0000

Data Index: 980940 (The Pile)

Max Activating Token Index: 442

Text #19

Max Range: 1.8373. Min Range: -1.8373

Max Act: 1.5253. Min Act: -0.0000

Data Index: 1616121 (The Pile)

Max Activating Token Index: 41