Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3067 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1447. Min Range: -1.1447

Max Act: 1.1447. Min Act: -0.0000

Data Index: 1342840 (The Pile)

Max Activating Token Index: 250

Text #1

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.9770. Min Act: -0.0000

Data Index: 1376461 (The Pile)

Max Activating Token Index: 438

Text #2

Max Range: 1.1447. Min Range: -1.1447

Max Act: 1.0226. Min Act: -0.0000

Data Index: 1657340 (The Pile)

Max Activating Token Index: 333

Text #3

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.9175. Min Act: -0.0000

Data Index: 1518722 (The Pile)

Max Activating Token Index: 501

Text #4

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8426. Min Act: -0.0000

Data Index: 386519 (The Pile)

Max Activating Token Index: 942

Text #5

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8002. Min Act: -0.0000

Data Index: 1538557 (The Pile)

Max Activating Token Index: 1018

Text #6

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8556. Min Act: -0.0000

Data Index: 307499 (The Pile)

Max Activating Token Index: 566

Text #7

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8426. Min Act: -0.0000

Data Index: 390100 (The Pile)

Max Activating Token Index: 539

Text #8

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8879. Min Act: -0.0000

Data Index: 990964 (The Pile)

Max Activating Token Index: 579

Text #9

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.9347. Min Act: -0.0000

Data Index: 993616 (The Pile)

Max Activating Token Index: 518

Text #10

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.7316. Min Act: -0.0000

Data Index: 412734 (The Pile)

Max Activating Token Index: 254

Text #11

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8169. Min Act: -0.0000

Data Index: 176267 (The Pile)

Max Activating Token Index: 342

Text #12

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8597. Min Act: -0.0000

Data Index: 588062 (The Pile)

Max Activating Token Index: 69

Text #13

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.7654. Min Act: -0.0000

Data Index: 1576542 (The Pile)

Max Activating Token Index: 346

Text #14

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8832. Min Act: -0.0000

Data Index: 1669932 (The Pile)

Max Activating Token Index: 987

Text #15

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.7577. Min Act: -0.0000

Data Index: 1578141 (The Pile)

Max Activating Token Index: 141

Text #16

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.7904. Min Act: -0.0000

Data Index: 1754021 (The Pile)

Max Activating Token Index: 608

Text #17

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8421. Min Act: -0.0000

Data Index: 1361806 (The Pile)

Max Activating Token Index: 35

Text #18

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.8320. Min Act: -0.0000

Data Index: 839537 (The Pile)

Max Activating Token Index: 50

Text #19

Max Range: 1.1447. Min Range: -1.1447

Max Act: 0.6674. Min Act: -0.0000

Data Index: 1803824 (The Pile)

Max Activating Token Index: 936