Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3836 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6806. Min Act: -0.0000

Data Index: 805136 (The Pile)

Max Activating Token Index: 625

Text #1

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.7165. Min Act: -0.0000

Data Index: 948832 (The Pile)

Max Activating Token Index: 133

Text #2

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6682. Min Act: -0.0000

Data Index: 1174859 (The Pile)

Max Activating Token Index: 372

Text #3

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6836. Min Act: -0.0000

Data Index: 1722334 (The Pile)

Max Activating Token Index: 874

Text #4

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6926. Min Act: -0.0000

Data Index: 235931 (The Pile)

Max Activating Token Index: 225

Text #5

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6481. Min Act: -0.0000

Data Index: 972689 (The Pile)

Max Activating Token Index: 165

Text #6

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.7081. Min Act: -0.0000

Data Index: 1956353 (The Pile)

Max Activating Token Index: 228

Text #7

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6566. Min Act: -0.0000

Data Index: 1234744 (The Pile)

Max Activating Token Index: 712

Text #8

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6393. Min Act: -0.0000

Data Index: 1810625 (The Pile)

Max Activating Token Index: 448

Text #9

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6108. Min Act: -0.0000

Data Index: 736035 (The Pile)

Max Activating Token Index: 1011

Text #10

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6469. Min Act: -0.0000

Data Index: 497116 (The Pile)

Max Activating Token Index: 282

Text #11

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6237. Min Act: -0.0000

Data Index: 1907964 (The Pile)

Max Activating Token Index: 910

Text #12

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6217. Min Act: -0.0000

Data Index: 866233 (The Pile)

Max Activating Token Index: 586

Text #13

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6001. Min Act: -0.0000

Data Index: 600012 (The Pile)

Max Activating Token Index: 324

Text #14

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6194. Min Act: -0.0000

Data Index: 1083068 (The Pile)

Max Activating Token Index: 196

Text #15

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.5807. Min Act: -0.0000

Data Index: 479859 (The Pile)

Max Activating Token Index: 24

Text #16

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6358. Min Act: -0.0000

Data Index: 1994638 (The Pile)

Max Activating Token Index: 499

Text #17

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.5733. Min Act: -0.0000

Data Index: 842481 (The Pile)

Max Activating Token Index: 679

Text #18

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6304. Min Act: -0.0000

Data Index: 894584 (The Pile)

Max Activating Token Index: 397

Text #19

Max Range: 0.7165. Min Range: -0.7165

Max Act: 0.6053. Min Act: -0.0000

Data Index: 1120460 (The Pile)

Max Activating Token Index: 478