Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2332 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.6047. Min Range: -5.6047

Max Act: 5.6047. Min Act: -0.0000

Data Index: 1481230 (The Pile)

Max Activating Token Index: 541

Text #1

Max Range: 5.6047. Min Range: -5.6047

Max Act: 5.0834. Min Act: -0.0000

Data Index: 402745 (The Pile)

Max Activating Token Index: 707

Text #2

Max Range: 5.6047. Min Range: -5.6047

Max Act: 5.0705. Min Act: -0.0000

Data Index: 1338973 (The Pile)

Max Activating Token Index: 839

Text #3

Max Range: 5.6047. Min Range: -5.6047

Max Act: 5.2239. Min Act: -0.0000

Data Index: 306539 (The Pile)

Max Activating Token Index: 861

Text #4

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.5921. Min Act: -0.0000

Data Index: 969227 (The Pile)

Max Activating Token Index: 760

Text #5

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.6181. Min Act: -0.0000

Data Index: 1539340 (The Pile)

Max Activating Token Index: 595

Text #6

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.7489. Min Act: -0.0000

Data Index: 1771468 (The Pile)

Max Activating Token Index: 330

Text #7

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.7074. Min Act: -0.0000

Data Index: 1032699 (The Pile)

Max Activating Token Index: 802

Text #8

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.5486. Min Act: -0.0000

Data Index: 1602337 (The Pile)

Max Activating Token Index: 847

Text #9

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.6802. Min Act: -0.0000

Data Index: 144509 (The Pile)

Max Activating Token Index: 646

Text #10

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.4986. Min Act: -0.0000

Data Index: 782145 (The Pile)

Max Activating Token Index: 194

Text #11

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.6353. Min Act: -0.0000

Data Index: 1733762 (The Pile)

Max Activating Token Index: 407

Text #12

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.3461. Min Act: -0.0000

Data Index: 1260734 (The Pile)

Max Activating Token Index: 281

Text #13

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.4467. Min Act: -0.0000

Data Index: 765020 (The Pile)

Max Activating Token Index: 543

Text #14

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.3690. Min Act: -0.0000

Data Index: 1262490 (The Pile)

Max Activating Token Index: 397

Text #15

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.1991. Min Act: -0.0000

Data Index: 1249535 (The Pile)

Max Activating Token Index: 911

Text #16

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.3671. Min Act: -0.0000

Data Index: 1567271 (The Pile)

Max Activating Token Index: 403

Text #17

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.3769. Min Act: -0.0000

Data Index: 1904473 (The Pile)

Max Activating Token Index: 921

Text #18

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.4296. Min Act: -0.0000

Data Index: 1788754 (The Pile)

Max Activating Token Index: 682

Text #19

Max Range: 5.6047. Min Range: -5.6047

Max Act: 4.2764. Min Act: -0.0000

Data Index: 754148 (The Pile)

Max Activating Token Index: 379