Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3906 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1228. Min Range: -1.1228

Max Act: 1.1228. Min Act: -0.0000

Data Index: 1050254 (The Pile)

Max Activating Token Index: 197

Text #1

Max Range: 1.1228. Min Range: -1.1228

Max Act: 1.0403. Min Act: -0.0000

Data Index: 1951319 (The Pile)

Max Activating Token Index: 318

Text #2

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.9720. Min Act: -0.0000

Data Index: 669517 (The Pile)

Max Activating Token Index: 214

Text #3

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.9503. Min Act: -0.0000

Data Index: 621406 (The Pile)

Max Activating Token Index: 519

Text #4

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.9351. Min Act: -0.0000

Data Index: 836161 (The Pile)

Max Activating Token Index: 342

Text #5

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.7770. Min Act: -0.0000

Data Index: 301740 (The Pile)

Max Activating Token Index: 907

Text #6

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8671. Min Act: -0.0000

Data Index: 1075218 (The Pile)

Max Activating Token Index: 632

Text #7

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.9155. Min Act: -0.0000

Data Index: 1977528 (The Pile)

Max Activating Token Index: 488

Text #8

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.9182. Min Act: -0.0000

Data Index: 1406861 (The Pile)

Max Activating Token Index: 457

Text #9

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8906. Min Act: -0.0000

Data Index: 753335 (The Pile)

Max Activating Token Index: 300

Text #10

Max Range: 1.1228. Min Range: -1.1228

Max Act: 1.0501. Min Act: -0.0000

Data Index: 424636 (The Pile)

Max Activating Token Index: 924

Text #11

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8431. Min Act: -0.0000

Data Index: 1821988 (The Pile)

Max Activating Token Index: 682

Text #12

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8530. Min Act: -0.0000

Data Index: 1453258 (The Pile)

Max Activating Token Index: 595

Text #13

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8932. Min Act: -0.0000

Data Index: 612220 (The Pile)

Max Activating Token Index: 477

Text #14

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8064. Min Act: -0.0000

Data Index: 1988566 (The Pile)

Max Activating Token Index: 669

Text #15

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8471. Min Act: -0.0000

Data Index: 1439752 (The Pile)

Max Activating Token Index: 489

Text #16

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.7843. Min Act: -0.0000

Data Index: 607116 (The Pile)

Max Activating Token Index: 866

Text #17

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8168. Min Act: -0.0000

Data Index: 1171608 (The Pile)

Max Activating Token Index: 115

Text #18

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.8289. Min Act: -0.0000

Data Index: 1888601 (The Pile)

Max Activating Token Index: 561

Text #19

Max Range: 1.1228. Min Range: -1.1228

Max Act: 0.7738. Min Act: -0.0000

Data Index: 569690 (The Pile)

Max Activating Token Index: 340