Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4727 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.1244. Min Act: -0.0000

Data Index: 189940 (The Pile)

Max Activating Token Index: 918

Text #1

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.1171. Min Act: -0.0000

Data Index: 614413 (The Pile)

Max Activating Token Index: 220

Text #2

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0788. Min Act: -0.0000

Data Index: 693777 (The Pile)

Max Activating Token Index: 520

Text #3

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0812. Min Act: -0.0000

Data Index: 829022 (The Pile)

Max Activating Token Index: 966

Text #4

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0792. Min Act: -0.0000

Data Index: 1514168 (The Pile)

Max Activating Token Index: 959

Text #5

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0735. Min Act: -0.0000

Data Index: 1783915 (The Pile)

Max Activating Token Index: 892

Text #6

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0730. Min Act: -0.0000

Data Index: 1103255 (The Pile)

Max Activating Token Index: 364

Text #7

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0698. Min Act: -0.0000

Data Index: 298514 (The Pile)

Max Activating Token Index: 991

Text #8

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0749. Min Act: -0.0000

Data Index: 785453 (The Pile)

Max Activating Token Index: 541

Text #9

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0687. Min Act: -0.0000

Data Index: 74499 (The Pile)

Max Activating Token Index: 1006

Text #10

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0657. Min Act: -0.0000

Data Index: 1808127 (The Pile)

Max Activating Token Index: 874

Text #11

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0703. Min Act: -0.0000

Data Index: 199759 (The Pile)

Max Activating Token Index: 969

Text #12

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0629. Min Act: -0.0000

Data Index: 250082 (The Pile)

Max Activating Token Index: 946

Text #13

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0676. Min Act: -0.0000

Data Index: 956853 (The Pile)

Max Activating Token Index: 902

Text #14

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0647. Min Act: -0.0000

Data Index: 1914739 (The Pile)

Max Activating Token Index: 788

Text #15

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0668. Min Act: -0.0000

Data Index: 1691359 (The Pile)

Max Activating Token Index: 813

Text #16

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0745. Min Act: -0.0000

Data Index: 1990610 (The Pile)

Max Activating Token Index: 682

Text #17

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0638. Min Act: -0.0000

Data Index: 1084799 (The Pile)

Max Activating Token Index: 581

Text #18

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0701. Min Act: -0.0000

Data Index: 803508 (The Pile)

Max Activating Token Index: 703

Text #19

Max Range: 0.1244. Min Range: -0.1244

Max Act: 0.0651. Min Act: -0.0000

Data Index: 87823 (The Pile)

Max Activating Token Index: 748