Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1189 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.8483. Min Act: -0.0000

Data Index: 317129 (The Pile)

Max Activating Token Index: 567

Text #1

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6222. Min Act: -0.0000

Data Index: 44057 (The Pile)

Max Activating Token Index: 336

Text #2

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6380. Min Act: -0.0000

Data Index: 134458 (The Pile)

Max Activating Token Index: 366

Text #3

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6625. Min Act: -0.0000

Data Index: 756702 (The Pile)

Max Activating Token Index: 272

Text #4

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6566. Min Act: -0.0000

Data Index: 1448418 (The Pile)

Max Activating Token Index: 484

Text #5

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5455. Min Act: -0.0000

Data Index: 1091369 (The Pile)

Max Activating Token Index: 171

Text #6

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6368. Min Act: -0.0000

Data Index: 1659356 (The Pile)

Max Activating Token Index: 245

Text #7

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5734. Min Act: -0.0000

Data Index: 1949026 (The Pile)

Max Activating Token Index: 502

Text #8

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6322. Min Act: -0.0000

Data Index: 1787013 (The Pile)

Max Activating Token Index: 145

Text #9

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6108. Min Act: -0.0000

Data Index: 465215 (The Pile)

Max Activating Token Index: 306

Text #10

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5911. Min Act: -0.0000

Data Index: 234649 (The Pile)

Max Activating Token Index: 503

Text #11

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6514. Min Act: -0.0000

Data Index: 617701 (The Pile)

Max Activating Token Index: 791

Text #12

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5852. Min Act: -0.0000

Data Index: 1289177 (The Pile)

Max Activating Token Index: 399

Text #13

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5662. Min Act: -0.0000

Data Index: 772694 (The Pile)

Max Activating Token Index: 673

Text #14

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5084. Min Act: -0.0000

Data Index: 1549214 (The Pile)

Max Activating Token Index: 586

Text #15

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.6507. Min Act: -0.0000

Data Index: 1079507 (The Pile)

Max Activating Token Index: 730

Text #16

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5921. Min Act: -0.0000

Data Index: 211460 (The Pile)

Max Activating Token Index: 69

Text #17

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5977. Min Act: -0.0000

Data Index: 1951023 (The Pile)

Max Activating Token Index: 142

Text #18

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5825. Min Act: -0.0000

Data Index: 105396 (The Pile)

Max Activating Token Index: 755

Text #19

Max Range: 0.8483. Min Range: -0.8483

Max Act: 0.5654. Min Act: -0.0000

Data Index: 1680028 (The Pile)

Max Activating Token Index: 207