Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2942 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.9968. Min Act: -0.0000

Data Index: 1825376 (The Pile)

Max Activating Token Index: 456

Text #1

Max Range: 5.1900. Min Range: -5.1900

Max Act: 5.1900. Min Act: -0.0000

Data Index: 1040481 (The Pile)

Max Activating Token Index: 640

Text #2

Max Range: 5.1900. Min Range: -5.1900

Max Act: 5.0224. Min Act: -0.0000

Data Index: 952399 (The Pile)

Max Activating Token Index: 501

Text #3

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.9690. Min Act: -0.0000

Data Index: 1793813 (The Pile)

Max Activating Token Index: 618

Text #4

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.4482. Min Act: -0.0000

Data Index: 514349 (The Pile)

Max Activating Token Index: 266

Text #5

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.7294. Min Act: -0.0000

Data Index: 666655 (The Pile)

Max Activating Token Index: 695

Text #6

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.7329. Min Act: -0.0000

Data Index: 1190250 (The Pile)

Max Activating Token Index: 413

Text #7

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.7476. Min Act: -0.0000

Data Index: 1676325 (The Pile)

Max Activating Token Index: 671

Text #8

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.7042. Min Act: -0.0000

Data Index: 1137421 (The Pile)

Max Activating Token Index: 475

Text #9

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.6085. Min Act: -0.0000

Data Index: 1513539 (The Pile)

Max Activating Token Index: 736

Text #10

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.6135. Min Act: -0.0000

Data Index: 584204 (The Pile)

Max Activating Token Index: 829

Text #11

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.4010. Min Act: -0.0000

Data Index: 1455091 (The Pile)

Max Activating Token Index: 237

Text #12

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.3673. Min Act: -0.0000

Data Index: 1604238 (The Pile)

Max Activating Token Index: 311

Text #13

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.3850. Min Act: -0.0000

Data Index: 1094737 (The Pile)

Max Activating Token Index: 277

Text #14

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.3216. Min Act: -0.0000

Data Index: 160550 (The Pile)

Max Activating Token Index: 344

Text #15

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.2549. Min Act: -0.0000

Data Index: 1997033 (The Pile)

Max Activating Token Index: 167

Text #16

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.0173. Min Act: -0.0000

Data Index: 964420 (The Pile)

Max Activating Token Index: 120

Text #17

Max Range: 5.1900. Min Range: -5.1900

Max Act: 3.9915. Min Act: -0.0000

Data Index: 1476612 (The Pile)

Max Activating Token Index: 210

Text #18

Max Range: 5.1900. Min Range: -5.1900

Max Act: 4.1863. Min Act: -0.0000

Data Index: 1701751 (The Pile)

Max Activating Token Index: 102

Text #19

Max Range: 5.1900. Min Range: -5.1900

Max Act: 3.8918. Min Act: -0.0000

Data Index: 1733545 (The Pile)

Max Activating Token Index: 498