Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4162 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3701. Min Range: -1.3701

Max Act: 1.3701. Min Act: -0.0000

Data Index: 1767723 (The Pile)

Max Activating Token Index: 984

Text #1

Max Range: 1.3701. Min Range: -1.3701

Max Act: 1.2374. Min Act: -0.0000

Data Index: 1114114 (The Pile)

Max Activating Token Index: 641

Text #2

Max Range: 1.3701. Min Range: -1.3701

Max Act: 1.0756. Min Act: -0.0000

Data Index: 1790671 (The Pile)

Max Activating Token Index: 680

Text #3

Max Range: 1.3701. Min Range: -1.3701

Max Act: 1.0601. Min Act: -0.0000

Data Index: 569063 (The Pile)

Max Activating Token Index: 1015

Text #4

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.9908. Min Act: -0.0000

Data Index: 260692 (The Pile)

Max Activating Token Index: 522

Text #5

Max Range: 1.3701. Min Range: -1.3701

Max Act: 1.0061. Min Act: -0.0000

Data Index: 1496260 (The Pile)

Max Activating Token Index: 335

Text #6

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.9751. Min Act: -0.0000

Data Index: 800272 (The Pile)

Max Activating Token Index: 682

Text #7

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.9282. Min Act: -0.0000

Data Index: 1528577 (The Pile)

Max Activating Token Index: 93

Text #8

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8773. Min Act: -0.0000

Data Index: 1505435 (The Pile)

Max Activating Token Index: 814

Text #9

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.9202. Min Act: -0.0000

Data Index: 1207032 (The Pile)

Max Activating Token Index: 16

Text #10

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8120. Min Act: -0.0000

Data Index: 5933 (The Pile)

Max Activating Token Index: 752

Text #11

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.7677. Min Act: -0.0000

Data Index: 1890783 (The Pile)

Max Activating Token Index: 908

Text #12

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.7844. Min Act: -0.0000

Data Index: 481212 (The Pile)

Max Activating Token Index: 43

Text #13

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8358. Min Act: -0.0000

Data Index: 1344742 (The Pile)

Max Activating Token Index: 890

Text #14

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8576. Min Act: -0.0000

Data Index: 260374 (The Pile)

Max Activating Token Index: 280

Text #15

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8543. Min Act: -0.0000

Data Index: 1042430 (The Pile)

Max Activating Token Index: 27

Text #16

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.7486. Min Act: -0.0000

Data Index: 222809 (The Pile)

Max Activating Token Index: 411

Text #17

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8227. Min Act: -0.0000

Data Index: 1193748 (The Pile)

Max Activating Token Index: 593

Text #18

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8319. Min Act: -0.0000

Data Index: 1460253 (The Pile)

Max Activating Token Index: 90

Text #19

Max Range: 1.3701. Min Range: -1.3701

Max Act: 0.8699. Min Act: -0.0000

Data Index: 1606773 (The Pile)

Max Activating Token Index: 722