Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3669 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.1665. Min Act: -0.0000

Data Index: 379410 (The Pile)

Max Activating Token Index: 178

Text #1

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.0781. Min Act: -0.0000

Data Index: 121662 (The Pile)

Max Activating Token Index: 88

Text #2

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.8506. Min Act: -0.0000

Data Index: 546057 (The Pile)

Max Activating Token Index: 669

Text #3

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.2102. Min Act: -0.0000

Data Index: 506834 (The Pile)

Max Activating Token Index: 690

Text #4

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.8812. Min Act: -0.0000

Data Index: 762794 (The Pile)

Max Activating Token Index: 883

Text #5

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.9398. Min Act: -0.0000

Data Index: 1211532 (The Pile)

Max Activating Token Index: 803

Text #6

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.1575. Min Act: -0.0000

Data Index: 977192 (The Pile)

Max Activating Token Index: 11

Text #7

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.9660. Min Act: -0.0000

Data Index: 1285838 (The Pile)

Max Activating Token Index: 37

Text #8

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.0210. Min Act: -0.0000

Data Index: 1725194 (The Pile)

Max Activating Token Index: 41

Text #9

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.9585. Min Act: -0.0000

Data Index: 360528 (The Pile)

Max Activating Token Index: 1011

Text #10

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.0111. Min Act: -0.0000

Data Index: 1511121 (The Pile)

Max Activating Token Index: 435

Text #11

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.0646. Min Act: -0.0000

Data Index: 210446 (The Pile)

Max Activating Token Index: 586

Text #12

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.9334. Min Act: -0.0000

Data Index: 1496920 (The Pile)

Max Activating Token Index: 8

Text #13

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.9755. Min Act: -0.0000

Data Index: 1763206 (The Pile)

Max Activating Token Index: 377

Text #14

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.9041. Min Act: -0.0000

Data Index: 542918 (The Pile)

Max Activating Token Index: 819

Text #15

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.9467. Min Act: -0.0000

Data Index: 966663 (The Pile)

Max Activating Token Index: 132

Text #16

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.0707. Min Act: -0.0000

Data Index: 238609 (The Pile)

Max Activating Token Index: 721

Text #17

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.8266. Min Act: -0.0000

Data Index: 668138 (The Pile)

Max Activating Token Index: 941

Text #18

Max Range: 1.2102. Min Range: -1.2102

Max Act: 1.0595. Min Act: -0.0000

Data Index: 1920646 (The Pile)

Max Activating Token Index: 618

Text #19

Max Range: 1.2102. Min Range: -1.2102

Max Act: 0.8405. Min Act: -0.0000

Data Index: 1221377 (The Pile)

Max Activating Token Index: 626