Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4492 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.9465. Min Act: -0.0000

Data Index: 1187405 (The Pile)

Max Activating Token Index: 243

Text #1

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.7654. Min Act: -0.0000

Data Index: 592851 (The Pile)

Max Activating Token Index: 489

Text #2

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.6712. Min Act: -0.0000

Data Index: 1821096 (The Pile)

Max Activating Token Index: 328

Text #3

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.6549. Min Act: -0.0000

Data Index: 977493 (The Pile)

Max Activating Token Index: 587

Text #4

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.4754. Min Act: -0.0000

Data Index: 1522769 (The Pile)

Max Activating Token Index: 333

Text #5

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.6498. Min Act: -0.0000

Data Index: 1703943 (The Pile)

Max Activating Token Index: 853

Text #6

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.4724. Min Act: -0.0000

Data Index: 1958481 (The Pile)

Max Activating Token Index: 236

Text #7

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.3037. Min Act: -0.0000

Data Index: 430282 (The Pile)

Max Activating Token Index: 576

Text #8

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.4323. Min Act: -0.0000

Data Index: 92357 (The Pile)

Max Activating Token Index: 734

Text #9

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.5844. Min Act: -0.0000

Data Index: 1572820 (The Pile)

Max Activating Token Index: 900

Text #10

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.2143. Min Act: -0.0000

Data Index: 1286078 (The Pile)

Max Activating Token Index: 564

Text #11

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.1682. Min Act: -0.0000

Data Index: 150489 (The Pile)

Max Activating Token Index: 893

Text #12

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.4687. Min Act: -0.0000

Data Index: 1967586 (The Pile)

Max Activating Token Index: 148

Text #13

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.3176. Min Act: -0.0000

Data Index: 380511 (The Pile)

Max Activating Token Index: 194

Text #14

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.1343. Min Act: -0.0000

Data Index: 603355 (The Pile)

Max Activating Token Index: 627

Text #15

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.2669. Min Act: -0.0000

Data Index: 96044 (The Pile)

Max Activating Token Index: 359

Text #16

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.1070. Min Act: -0.0000

Data Index: 1845561 (The Pile)

Max Activating Token Index: 560

Text #17

Max Range: 3.9465. Min Range: -3.9465

Max Act: 2.9895. Min Act: -0.0000

Data Index: 1141234 (The Pile)

Max Activating Token Index: 570

Text #18

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.2150. Min Act: -0.0000

Data Index: 1204725 (The Pile)

Max Activating Token Index: 429

Text #19

Max Range: 3.9465. Min Range: -3.9465

Max Act: 3.2071. Min Act: -0.0000

Data Index: 1025853 (The Pile)

Max Activating Token Index: 523