Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3181 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.6333. Min Act: -0.0000

Data Index: 17288 (The Pile)

Max Activating Token Index: 491

Text #1

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.5134. Min Act: -0.0000

Data Index: 1348205 (The Pile)

Max Activating Token Index: 394

Text #2

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.1773. Min Act: -0.0000

Data Index: 256080 (The Pile)

Max Activating Token Index: 713

Text #3

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.3088. Min Act: -0.0000

Data Index: 453033 (The Pile)

Max Activating Token Index: 897

Text #4

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.3289. Min Act: -0.0000

Data Index: 757484 (The Pile)

Max Activating Token Index: 770

Text #5

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.1712. Min Act: -0.0000

Data Index: 287205 (The Pile)

Max Activating Token Index: 965

Text #6

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.0306. Min Act: -0.0000

Data Index: 127848 (The Pile)

Max Activating Token Index: 732

Text #7

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.2291. Min Act: -0.0000

Data Index: 704685 (The Pile)

Max Activating Token Index: 379

Text #8

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.3335. Min Act: -0.0000

Data Index: 171521 (The Pile)

Max Activating Token Index: 1000

Text #9

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.3308. Min Act: -0.0000

Data Index: 1075850 (The Pile)

Max Activating Token Index: 714

Text #10

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.1878. Min Act: -0.0000

Data Index: 797524 (The Pile)

Max Activating Token Index: 33

Text #11

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.2634. Min Act: -0.0000

Data Index: 438762 (The Pile)

Max Activating Token Index: 633

Text #12

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.2520. Min Act: -0.0000

Data Index: 718455 (The Pile)

Max Activating Token Index: 627

Text #13

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.1055. Min Act: -0.0000

Data Index: 1723047 (The Pile)

Max Activating Token Index: 674

Text #14

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.0898. Min Act: -0.0000

Data Index: 835853 (The Pile)

Max Activating Token Index: 763

Text #15

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.1397. Min Act: -0.0000

Data Index: 1944089 (The Pile)

Max Activating Token Index: 710

Text #16

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.2869. Min Act: -0.0000

Data Index: 113443 (The Pile)

Max Activating Token Index: 716

Text #17

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.1410. Min Act: -0.0000

Data Index: 1650589 (The Pile)

Max Activating Token Index: 923

Text #18

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.1742. Min Act: -0.0000

Data Index: 1319122 (The Pile)

Max Activating Token Index: 956

Text #19

Max Range: 1.6333. Min Range: -1.6333

Max Act: 1.3302. Min Act: -0.0000

Data Index: 887327 (The Pile)

Max Activating Token Index: 896