Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 226 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.3718. Min Range: -1.3718

Max Act: 1.3718. Min Act: -0.0000

Data Index: 275971 (The Pile)

Max Activating Token Index: 255

Text #1

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9979. Min Act: -0.0000

Data Index: 1338746 (The Pile)

Max Activating Token Index: 711

Text #2

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9927. Min Act: -0.0000

Data Index: 1594417 (The Pile)

Max Activating Token Index: 845

Text #3

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9720. Min Act: -0.0000

Data Index: 1520002 (The Pile)

Max Activating Token Index: 155

Text #4

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9671. Min Act: -0.0000

Data Index: 1328284 (The Pile)

Max Activating Token Index: 996

Text #5

Max Range: 1.3718. Min Range: -1.3718

Max Act: 1.0863. Min Act: -0.0000

Data Index: 1665939 (The Pile)

Max Activating Token Index: 833

Text #6

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9508. Min Act: -0.0000

Data Index: 1726312 (The Pile)

Max Activating Token Index: 92

Text #7

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9380. Min Act: -0.0000

Data Index: 315882 (The Pile)

Max Activating Token Index: 315

Text #8

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9832. Min Act: -0.0000

Data Index: 1451764 (The Pile)

Max Activating Token Index: 518

Text #9

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9711. Min Act: -0.0000

Data Index: 1402004 (The Pile)

Max Activating Token Index: 1020

Text #10

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9750. Min Act: -0.0000

Data Index: 1242373 (The Pile)

Max Activating Token Index: 351

Text #11

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9590. Min Act: -0.0000

Data Index: 1143159 (The Pile)

Max Activating Token Index: 345

Text #12

Max Range: 1.3718. Min Range: -1.3718

Max Act: 1.0121. Min Act: -0.0000

Data Index: 1819459 (The Pile)

Max Activating Token Index: 850

Text #13

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.8896. Min Act: -0.0000

Data Index: 1058201 (The Pile)

Max Activating Token Index: 1003

Text #14

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.8725. Min Act: -0.0000

Data Index: 822576 (The Pile)

Max Activating Token Index: 668

Text #15

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9173. Min Act: -0.0000

Data Index: 1408188 (The Pile)

Max Activating Token Index: 374

Text #16

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9809. Min Act: -0.0000

Data Index: 250889 (The Pile)

Max Activating Token Index: 535

Text #17

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9140. Min Act: -0.0000

Data Index: 1099406 (The Pile)

Max Activating Token Index: 379

Text #18

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.9076. Min Act: -0.0000

Data Index: 1761886 (The Pile)

Max Activating Token Index: 796

Text #19

Max Range: 1.3718. Min Range: -1.3718

Max Act: 0.8578. Min Act: -0.0000

Data Index: 571929 (The Pile)

Max Activating Token Index: 192