Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3541 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.6130. Min Act: -0.0000

Data Index: 253697 (The Pile)

Max Activating Token Index: 895

Text #1

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5976. Min Act: -0.0000

Data Index: 666425 (The Pile)

Max Activating Token Index: 491

Text #2

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5615. Min Act: -0.0000

Data Index: 1278811 (The Pile)

Max Activating Token Index: 661

Text #3

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5606. Min Act: -0.0000

Data Index: 891763 (The Pile)

Max Activating Token Index: 518

Text #4

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5904. Min Act: -0.0000

Data Index: 1610019 (The Pile)

Max Activating Token Index: 388

Text #5

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5719. Min Act: -0.0000

Data Index: 1848531 (The Pile)

Max Activating Token Index: 444

Text #6

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5509. Min Act: -0.0000

Data Index: 823196 (The Pile)

Max Activating Token Index: 703

Text #7

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5481. Min Act: -0.0000

Data Index: 1567810 (The Pile)

Max Activating Token Index: 678

Text #8

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5456. Min Act: -0.0000

Data Index: 1247448 (The Pile)

Max Activating Token Index: 185

Text #9

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5435. Min Act: -0.0000

Data Index: 839106 (The Pile)

Max Activating Token Index: 673

Text #10

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5606. Min Act: -0.0000

Data Index: 1320405 (The Pile)

Max Activating Token Index: 698

Text #11

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5553. Min Act: -0.0000

Data Index: 1979190 (The Pile)

Max Activating Token Index: 986

Text #12

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5513. Min Act: -0.0000

Data Index: 1942325 (The Pile)

Max Activating Token Index: 314

Text #13

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5315. Min Act: -0.0000

Data Index: 43044 (The Pile)

Max Activating Token Index: 566

Text #14

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5351. Min Act: -0.0000

Data Index: 975622 (The Pile)

Max Activating Token Index: 911

Text #15

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5482. Min Act: -0.0000

Data Index: 124391 (The Pile)

Max Activating Token Index: 245

Text #16

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5312. Min Act: -0.0000

Data Index: 982935 (The Pile)

Max Activating Token Index: 469

Text #17

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5391. Min Act: -0.0000

Data Index: 1897854 (The Pile)

Max Activating Token Index: 792

Text #18

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5163. Min Act: -0.0000

Data Index: 1125510 (The Pile)

Max Activating Token Index: 1023

Text #19

Max Range: 0.6130. Min Range: -0.6130

Max Act: 0.5253. Min Act: -0.0000

Data Index: 1786413 (The Pile)

Max Activating Token Index: 370