Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2181 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0267. Min Range: -1.0267

Max Act: 1.0267. Min Act: -0.0000

Data Index: 1700451 (The Pile)

Max Activating Token Index: 660

Text #1

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7508. Min Act: -0.0000

Data Index: 1127679 (The Pile)

Max Activating Token Index: 341

Text #2

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7957. Min Act: -0.0000

Data Index: 212870 (The Pile)

Max Activating Token Index: 594

Text #3

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7850. Min Act: -0.0000

Data Index: 62087 (The Pile)

Max Activating Token Index: 379

Text #4

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7644. Min Act: -0.0000

Data Index: 1331787 (The Pile)

Max Activating Token Index: 793

Text #5

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7238. Min Act: -0.0000

Data Index: 854687 (The Pile)

Max Activating Token Index: 839

Text #6

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7424. Min Act: -0.0000

Data Index: 1109617 (The Pile)

Max Activating Token Index: 699

Text #7

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6467. Min Act: -0.0000

Data Index: 1165307 (The Pile)

Max Activating Token Index: 673

Text #8

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7805. Min Act: -0.0000

Data Index: 17461 (The Pile)

Max Activating Token Index: 531

Text #9

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6755. Min Act: -0.0000

Data Index: 616027 (The Pile)

Max Activating Token Index: 632

Text #10

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7337. Min Act: -0.0000

Data Index: 71131 (The Pile)

Max Activating Token Index: 188

Text #11

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7111. Min Act: -0.0000

Data Index: 1807484 (The Pile)

Max Activating Token Index: 100

Text #12

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.7046. Min Act: -0.0000

Data Index: 1264199 (The Pile)

Max Activating Token Index: 217

Text #13

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6507. Min Act: -0.0000

Data Index: 125684 (The Pile)

Max Activating Token Index: 184

Text #14

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6816. Min Act: -0.0000

Data Index: 841552 (The Pile)

Max Activating Token Index: 838

Text #15

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6293. Min Act: -0.0000

Data Index: 326045 (The Pile)

Max Activating Token Index: 259

Text #16

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6460. Min Act: -0.0000

Data Index: 1016631 (The Pile)

Max Activating Token Index: 327

Text #17

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6418. Min Act: -0.0000

Data Index: 919438 (The Pile)

Max Activating Token Index: 920

Text #18

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6594. Min Act: -0.0000

Data Index: 1966312 (The Pile)

Max Activating Token Index: 420

Text #19

Max Range: 1.0267. Min Range: -1.0267

Max Act: 0.6966. Min Act: -0.0000

Data Index: 553046 (The Pile)

Max Activating Token Index: 218