Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 919 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.2857. Min Range: -1.2857

Max Act: 1.2857. Min Act: -0.0000

Data Index: 1086098 (The Pile)

Max Activating Token Index: 322

Text #1

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9761. Min Act: -0.0000

Data Index: 1930320 (The Pile)

Max Activating Token Index: 967

Text #2

Max Range: 1.2857. Min Range: -1.2857

Max Act: 1.0033. Min Act: -0.0000

Data Index: 693046 (The Pile)

Max Activating Token Index: 738

Text #3

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9905. Min Act: -0.0000

Data Index: 1249366 (The Pile)

Max Activating Token Index: 188

Text #4

Max Range: 1.2857. Min Range: -1.2857

Max Act: 1.0053. Min Act: -0.0000

Data Index: 1172282 (The Pile)

Max Activating Token Index: 79

Text #5

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9859. Min Act: -0.0000

Data Index: 1185749 (The Pile)

Max Activating Token Index: 927

Text #6

Max Range: 1.2857. Min Range: -1.2857

Max Act: 1.0838. Min Act: -0.0000

Data Index: 1732926 (The Pile)

Max Activating Token Index: 820

Text #7

Max Range: 1.2857. Min Range: -1.2857

Max Act: 1.0109. Min Act: -0.0000

Data Index: 783696 (The Pile)

Max Activating Token Index: 632

Text #8

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9432. Min Act: -0.0000

Data Index: 636768 (The Pile)

Max Activating Token Index: 772

Text #9

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9583. Min Act: -0.0000

Data Index: 1530445 (The Pile)

Max Activating Token Index: 366

Text #10

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9520. Min Act: -0.0000

Data Index: 937381 (The Pile)

Max Activating Token Index: 957

Text #11

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9154. Min Act: -0.0000

Data Index: 1751238 (The Pile)

Max Activating Token Index: 67

Text #12

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9606. Min Act: -0.0000

Data Index: 1390496 (The Pile)

Max Activating Token Index: 889

Text #13

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.8796. Min Act: -0.0000

Data Index: 1386005 (The Pile)

Max Activating Token Index: 808

Text #14

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.8006. Min Act: -0.0000

Data Index: 456725 (The Pile)

Max Activating Token Index: 104

Text #15

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9632. Min Act: -0.0000

Data Index: 847713 (The Pile)

Max Activating Token Index: 85

Text #16

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.8926. Min Act: -0.0000

Data Index: 508858 (The Pile)

Max Activating Token Index: 66

Text #17

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.9160. Min Act: -0.0000

Data Index: 176302 (The Pile)

Max Activating Token Index: 135

Text #18

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.8577. Min Act: -0.0000

Data Index: 1265878 (The Pile)

Max Activating Token Index: 1007

Text #19

Max Range: 1.2857. Min Range: -1.2857

Max Act: 0.8473. Min Act: -0.0000

Data Index: 1266120 (The Pile)

Max Activating Token Index: 267