Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2659 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.6031. Min Act: -0.0000

Data Index: 428529 (The Pile)

Max Activating Token Index: 549

Text #1

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5711. Min Act: -0.0000

Data Index: 1325704 (The Pile)

Max Activating Token Index: 378

Text #2

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5646. Min Act: -0.0000

Data Index: 802875 (The Pile)

Max Activating Token Index: 592

Text #3

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.6089. Min Act: -0.0000

Data Index: 958961 (The Pile)

Max Activating Token Index: 833

Text #4

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5240. Min Act: -0.0000

Data Index: 1531653 (The Pile)

Max Activating Token Index: 355

Text #5

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4938. Min Act: -0.0000

Data Index: 1163221 (The Pile)

Max Activating Token Index: 800

Text #6

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5106. Min Act: -0.0000

Data Index: 440388 (The Pile)

Max Activating Token Index: 39

Text #7

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4791. Min Act: -0.0000

Data Index: 39516 (The Pile)

Max Activating Token Index: 847

Text #8

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4799. Min Act: -0.0000

Data Index: 451580 (The Pile)

Max Activating Token Index: 848

Text #9

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4935. Min Act: -0.0000

Data Index: 481017 (The Pile)

Max Activating Token Index: 277

Text #10

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5096. Min Act: -0.0000

Data Index: 353475 (The Pile)

Max Activating Token Index: 549

Text #11

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5178. Min Act: -0.0000

Data Index: 1456115 (The Pile)

Max Activating Token Index: 925

Text #12

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5131. Min Act: -0.0000

Data Index: 1455042 (The Pile)

Max Activating Token Index: 937

Text #13

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4793. Min Act: -0.0000

Data Index: 1486778 (The Pile)

Max Activating Token Index: 523

Text #14

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4747. Min Act: -0.0000

Data Index: 1176079 (The Pile)

Max Activating Token Index: 949

Text #15

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4482. Min Act: -0.0000

Data Index: 785689 (The Pile)

Max Activating Token Index: 600

Text #16

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4194. Min Act: -0.0000

Data Index: 195647 (The Pile)

Max Activating Token Index: 687

Text #17

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4439. Min Act: -0.0000

Data Index: 1117193 (The Pile)

Max Activating Token Index: 791

Text #18

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.4518. Min Act: -0.0000

Data Index: 837968 (The Pile)

Max Activating Token Index: 41

Text #19

Max Range: 0.6089. Min Range: -0.6089

Max Act: 0.5020. Min Act: -0.0000

Data Index: 1224909 (The Pile)

Max Activating Token Index: 709