Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2934 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.9071. Min Range: -5.9071

Max Act: 5.8929. Min Act: -0.0000

Data Index: 1996169 (The Pile)

Max Activating Token Index: 351

Text #1

Max Range: 5.9071. Min Range: -5.9071

Max Act: 5.9071. Min Act: -0.0000

Data Index: 1028022 (The Pile)

Max Activating Token Index: 673

Text #2

Max Range: 5.9071. Min Range: -5.9071

Max Act: 5.8833. Min Act: -0.0000

Data Index: 1172103 (The Pile)

Max Activating Token Index: 409

Text #3

Max Range: 5.9071. Min Range: -5.9071

Max Act: 5.1931. Min Act: -0.0000

Data Index: 419456 (The Pile)

Max Activating Token Index: 483

Text #4

Max Range: 5.9071. Min Range: -5.9071

Max Act: 5.2738. Min Act: -0.0000

Data Index: 588635 (The Pile)

Max Activating Token Index: 509

Text #5

Max Range: 5.9071. Min Range: -5.9071

Max Act: 5.7073. Min Act: -0.0000

Data Index: 890370 (The Pile)

Max Activating Token Index: 926

Text #6

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.6669. Min Act: -0.0000

Data Index: 1032012 (The Pile)

Max Activating Token Index: 453

Text #7

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.9914. Min Act: -0.0000

Data Index: 994299 (The Pile)

Max Activating Token Index: 477

Text #8

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.8044. Min Act: -0.0000

Data Index: 1839569 (The Pile)

Max Activating Token Index: 103

Text #9

Max Range: 5.9071. Min Range: -5.9071

Max Act: 5.1257. Min Act: -0.0000

Data Index: 1852001 (The Pile)

Max Activating Token Index: 275

Text #10

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.8138. Min Act: -0.0000

Data Index: 1325891 (The Pile)

Max Activating Token Index: 476

Text #11

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.6033. Min Act: -0.0000

Data Index: 1704081 (The Pile)

Max Activating Token Index: 735

Text #12

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.8435. Min Act: -0.0000

Data Index: 200398 (The Pile)

Max Activating Token Index: 98

Text #13

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.8093. Min Act: -0.0000

Data Index: 1252544 (The Pile)

Max Activating Token Index: 506

Text #14

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.6183. Min Act: -0.0000

Data Index: 1156010 (The Pile)

Max Activating Token Index: 312

Text #15

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.7862. Min Act: -0.0000

Data Index: 1685275 (The Pile)

Max Activating Token Index: 284

Text #16

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.6137. Min Act: -0.0000

Data Index: 16468 (The Pile)

Max Activating Token Index: 574

Text #17

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.6399. Min Act: -0.0000

Data Index: 288839 (The Pile)

Max Activating Token Index: 257

Text #18

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.1618. Min Act: -0.0000

Data Index: 1661200 (The Pile)

Max Activating Token Index: 836

Text #19

Max Range: 5.9071. Min Range: -5.9071

Max Act: 4.7650. Min Act: -0.0000

Data Index: 1032355 (The Pile)

Max Activating Token Index: 185