Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4171 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.7126. Min Act: -0.0000

Data Index: 332832 (The Pile)

Max Activating Token Index: 413

Text #1

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.6486. Min Act: -0.0000

Data Index: 234603 (The Pile)

Max Activating Token Index: 735

Text #2

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.6526. Min Act: -0.0000

Data Index: 1434750 (The Pile)

Max Activating Token Index: 311

Text #3

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.3327. Min Act: -0.0000

Data Index: 1180808 (The Pile)

Max Activating Token Index: 839

Text #4

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.4367. Min Act: -0.0000

Data Index: 1968440 (The Pile)

Max Activating Token Index: 466

Text #5

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2337. Min Act: -0.0000

Data Index: 1351034 (The Pile)

Max Activating Token Index: 938

Text #6

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.6385. Min Act: -0.0000

Data Index: 30823 (The Pile)

Max Activating Token Index: 834

Text #7

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.4005. Min Act: -0.0000

Data Index: 1441395 (The Pile)

Max Activating Token Index: 561

Text #8

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2344. Min Act: -0.0000

Data Index: 1779568 (The Pile)

Max Activating Token Index: 700

Text #9

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2846. Min Act: -0.0000

Data Index: 175563 (The Pile)

Max Activating Token Index: 131

Text #10

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2986. Min Act: -0.0000

Data Index: 431683 (The Pile)

Max Activating Token Index: 208

Text #11

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.4919. Min Act: -0.0000

Data Index: 1670542 (The Pile)

Max Activating Token Index: 365

Text #12

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.3090. Min Act: -0.0000

Data Index: 1936878 (The Pile)

Max Activating Token Index: 226

Text #13

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2764. Min Act: -0.0000

Data Index: 411952 (The Pile)

Max Activating Token Index: 815

Text #14

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2208. Min Act: -0.0000

Data Index: 113566 (The Pile)

Max Activating Token Index: 577

Text #15

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2922. Min Act: -0.0000

Data Index: 108360 (The Pile)

Max Activating Token Index: 831

Text #16

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2859. Min Act: -0.0000

Data Index: 1537576 (The Pile)

Max Activating Token Index: 843

Text #17

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2191. Min Act: -0.0000

Data Index: 1222270 (The Pile)

Max Activating Token Index: 893

Text #18

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2817. Min Act: -0.0000

Data Index: 1169506 (The Pile)

Max Activating Token Index: 218

Text #19

Max Range: 1.7126. Min Range: -1.7126

Max Act: 1.2626. Min Act: -0.0000

Data Index: 1849895 (The Pile)

Max Activating Token Index: 298