Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2686 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.6642. Min Act: -0.0000

Data Index: 778134 (The Pile)

Max Activating Token Index: 56

Text #1

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.6339. Min Act: -0.0000

Data Index: 992052 (The Pile)

Max Activating Token Index: 986

Text #2

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.6476. Min Act: -0.0000

Data Index: 270967 (The Pile)

Max Activating Token Index: 652

Text #3

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5434. Min Act: -0.0000

Data Index: 1571038 (The Pile)

Max Activating Token Index: 936

Text #4

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.6088. Min Act: -0.0000

Data Index: 465855 (The Pile)

Max Activating Token Index: 501

Text #5

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5882. Min Act: -0.0000

Data Index: 953577 (The Pile)

Max Activating Token Index: 771

Text #6

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5104. Min Act: -0.0000

Data Index: 1824218 (The Pile)

Max Activating Token Index: 842

Text #7

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.6029. Min Act: -0.0000

Data Index: 1934895 (The Pile)

Max Activating Token Index: 737

Text #8

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5373. Min Act: -0.0000

Data Index: 312838 (The Pile)

Max Activating Token Index: 1004

Text #9

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.6157. Min Act: -0.0000

Data Index: 112114 (The Pile)

Max Activating Token Index: 156

Text #10

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5390. Min Act: -0.0000

Data Index: 1885980 (The Pile)

Max Activating Token Index: 586

Text #11

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.6016. Min Act: -0.0000

Data Index: 653287 (The Pile)

Max Activating Token Index: 695

Text #12

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5466. Min Act: -0.0000

Data Index: 1913316 (The Pile)

Max Activating Token Index: 954

Text #13

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.4905. Min Act: -0.0000

Data Index: 1829058 (The Pile)

Max Activating Token Index: 679

Text #14

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5304. Min Act: -0.0000

Data Index: 1499810 (The Pile)

Max Activating Token Index: 920

Text #15

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5396. Min Act: -0.0000

Data Index: 968027 (The Pile)

Max Activating Token Index: 41

Text #16

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5316. Min Act: -0.0000

Data Index: 666949 (The Pile)

Max Activating Token Index: 973

Text #17

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5265. Min Act: -0.0000

Data Index: 708396 (The Pile)

Max Activating Token Index: 864

Text #18

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5706. Min Act: -0.0000

Data Index: 1717837 (The Pile)

Max Activating Token Index: 238

Text #19

Max Range: 0.6642. Min Range: -0.6642

Max Act: 0.5145. Min Act: -0.0000

Data Index: 1356101 (The Pile)

Max Activating Token Index: 120