Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 5115 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9737. Min Act: -0.0000

Data Index: 1565513 (The Pile)

Max Activating Token Index: 401

Text #1

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9545. Min Act: -0.0000

Data Index: 985813 (The Pile)

Max Activating Token Index: 293

Text #2

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9000. Min Act: -0.0000

Data Index: 194304 (The Pile)

Max Activating Token Index: 656

Text #3

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8898. Min Act: -0.0000

Data Index: 1249103 (The Pile)

Max Activating Token Index: 922

Text #4

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9401. Min Act: -0.0000

Data Index: 132148 (The Pile)

Max Activating Token Index: 504

Text #5

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8283. Min Act: -0.0000

Data Index: 612131 (The Pile)

Max Activating Token Index: 291

Text #6

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8947. Min Act: -0.0000

Data Index: 1781734 (The Pile)

Max Activating Token Index: 474

Text #7

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9243. Min Act: -0.0000

Data Index: 68255 (The Pile)

Max Activating Token Index: 480

Text #8

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8925. Min Act: -0.0000

Data Index: 1697365 (The Pile)

Max Activating Token Index: 291

Text #9

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8184. Min Act: -0.0000

Data Index: 1861567 (The Pile)

Max Activating Token Index: 718

Text #10

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9273. Min Act: -0.0000

Data Index: 1648377 (The Pile)

Max Activating Token Index: 515

Text #11

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8627. Min Act: -0.0000

Data Index: 1654941 (The Pile)

Max Activating Token Index: 314

Text #12

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9167. Min Act: -0.0000

Data Index: 521082 (The Pile)

Max Activating Token Index: 58

Text #13

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8537. Min Act: -0.0000

Data Index: 1840311 (The Pile)

Max Activating Token Index: 833

Text #14

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8398. Min Act: -0.0000

Data Index: 814447 (The Pile)

Max Activating Token Index: 286

Text #15

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8242. Min Act: -0.0000

Data Index: 414741 (The Pile)

Max Activating Token Index: 614

Text #16

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.9141. Min Act: -0.0000

Data Index: 1725045 (The Pile)

Max Activating Token Index: 1016

Text #17

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.7896. Min Act: -0.0000

Data Index: 763537 (The Pile)

Max Activating Token Index: 833

Text #18

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.7885. Min Act: -0.0000

Data Index: 1268251 (The Pile)

Max Activating Token Index: 959

Text #19

Max Range: 0.9737. Min Range: -0.9737

Max Act: 0.8192. Min Act: -0.0000

Data Index: 981256 (The Pile)

Max Activating Token Index: 406