Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 867 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.9345. Min Act: -0.0000

Data Index: 203222 (The Pile)

Max Activating Token Index: 319

Text #1

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.6413. Min Act: -0.0000

Data Index: 1506347 (The Pile)

Max Activating Token Index: 192

Text #2

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.9949. Min Act: -0.0000

Data Index: 869767 (The Pile)

Max Activating Token Index: 735

Text #3

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.6701. Min Act: -0.0000

Data Index: 1919889 (The Pile)

Max Activating Token Index: 546

Text #4

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.8281. Min Act: -0.0000

Data Index: 1158035 (The Pile)

Max Activating Token Index: 170

Text #5

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.6577. Min Act: -0.0000

Data Index: 212644 (The Pile)

Max Activating Token Index: 925

Text #6

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.4743. Min Act: -0.0000

Data Index: 605574 (The Pile)

Max Activating Token Index: 156

Text #7

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.3278. Min Act: -0.0000

Data Index: 270014 (The Pile)

Max Activating Token Index: 531

Text #8

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.1653. Min Act: -0.0000

Data Index: 1030972 (The Pile)

Max Activating Token Index: 325

Text #9

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.1183. Min Act: -0.0000

Data Index: 1918007 (The Pile)

Max Activating Token Index: 546

Text #10

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.1130. Min Act: -0.0000

Data Index: 953957 (The Pile)

Max Activating Token Index: 344

Text #11

Max Range: 5.9949. Min Range: -5.9949

Max Act: 4.9092. Min Act: -0.0000

Data Index: 1624865 (The Pile)

Max Activating Token Index: 590

Text #12

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.2512. Min Act: -0.0000

Data Index: 2974 (The Pile)

Max Activating Token Index: 76

Text #13

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.0845. Min Act: -0.0000

Data Index: 1214461 (The Pile)

Max Activating Token Index: 512

Text #14

Max Range: 5.9949. Min Range: -5.9949

Max Act: 4.8781. Min Act: -0.0000

Data Index: 1637042 (The Pile)

Max Activating Token Index: 934

Text #15

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.0845. Min Act: -0.0000

Data Index: 1416155 (The Pile)

Max Activating Token Index: 825

Text #16

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.2037. Min Act: -0.0000

Data Index: 287467 (The Pile)

Max Activating Token Index: 57

Text #17

Max Range: 5.9949. Min Range: -5.9949

Max Act: 5.1903. Min Act: -0.0000

Data Index: 469315 (The Pile)

Max Activating Token Index: 139

Text #18

Max Range: 5.9949. Min Range: -5.9949

Max Act: 4.8075. Min Act: -0.0000

Data Index: 1984839 (The Pile)

Max Activating Token Index: 356

Text #19

Max Range: 5.9949. Min Range: -5.9949

Max Act: 4.9795. Min Act: -0.0000

Data Index: 148120 (The Pile)

Max Activating Token Index: 1005