Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2521 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.9439. Min Act: -0.0000

Data Index: 1736571 (The Pile)

Max Activating Token Index: 947

Text #1

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.7426. Min Act: -0.0000

Data Index: 147171 (The Pile)

Max Activating Token Index: 175

Text #2

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.7521. Min Act: -0.0000

Data Index: 489017 (The Pile)

Max Activating Token Index: 952

Text #3

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.8672. Min Act: -0.0000

Data Index: 373398 (The Pile)

Max Activating Token Index: 84

Text #4

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.6909. Min Act: -0.0000

Data Index: 1747446 (The Pile)

Max Activating Token Index: 368

Text #5

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.6194. Min Act: -0.0000

Data Index: 1052921 (The Pile)

Max Activating Token Index: 953

Text #6

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.5668. Min Act: -0.0000

Data Index: 80810 (The Pile)

Max Activating Token Index: 489

Text #7

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.5541. Min Act: -0.0000

Data Index: 195098 (The Pile)

Max Activating Token Index: 455

Text #8

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.5854. Min Act: -0.0000

Data Index: 1600973 (The Pile)

Max Activating Token Index: 139

Text #9

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.5332. Min Act: -0.0000

Data Index: 1113153 (The Pile)

Max Activating Token Index: 1019

Text #10

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.6005. Min Act: -0.0000

Data Index: 1612494 (The Pile)

Max Activating Token Index: 102

Text #11

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.4904. Min Act: -0.0000

Data Index: 1738993 (The Pile)

Max Activating Token Index: 242

Text #12

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.6014. Min Act: -0.0000

Data Index: 1397231 (The Pile)

Max Activating Token Index: 193

Text #13

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.4963. Min Act: -0.0000

Data Index: 180840 (The Pile)

Max Activating Token Index: 19

Text #14

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.5071. Min Act: -0.0000

Data Index: 871441 (The Pile)

Max Activating Token Index: 13

Text #15

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.4992. Min Act: -0.0000

Data Index: 716248 (The Pile)

Max Activating Token Index: 46

Text #16

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.4611. Min Act: -0.0000

Data Index: 40008 (The Pile)

Max Activating Token Index: 710

Text #17

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.4490. Min Act: -0.0000

Data Index: 1896235 (The Pile)

Max Activating Token Index: 580

Text #18

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.4611. Min Act: -0.0000

Data Index: 785315 (The Pile)

Max Activating Token Index: 233

Text #19

Max Range: 0.9439. Min Range: -0.9439

Max Act: 0.4732. Min Act: -0.0000

Data Index: 1423976 (The Pile)

Max Activating Token Index: 95