Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2651 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1978. Min Range: -1.1978

Max Act: 1.1978. Min Act: -0.0000

Data Index: 21075 (The Pile)

Max Activating Token Index: 446

Text #1

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.9172. Min Act: -0.0000

Data Index: 1872900 (The Pile)

Max Activating Token Index: 317

Text #2

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.8754. Min Act: -0.0000

Data Index: 1923385 (The Pile)

Max Activating Token Index: 430

Text #3

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.8707. Min Act: -0.0000

Data Index: 753473 (The Pile)

Max Activating Token Index: 200

Text #4

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.8230. Min Act: -0.0000

Data Index: 1874279 (The Pile)

Max Activating Token Index: 388

Text #5

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.8467. Min Act: -0.0000

Data Index: 77140 (The Pile)

Max Activating Token Index: 363

Text #6

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7477. Min Act: -0.0000

Data Index: 1214766 (The Pile)

Max Activating Token Index: 273

Text #7

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.8043. Min Act: -0.0000

Data Index: 842073 (The Pile)

Max Activating Token Index: 855

Text #8

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7896. Min Act: -0.0000

Data Index: 895878 (The Pile)

Max Activating Token Index: 899

Text #9

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7905. Min Act: -0.0000

Data Index: 1240146 (The Pile)

Max Activating Token Index: 203

Text #10

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7837. Min Act: -0.0000

Data Index: 270918 (The Pile)

Max Activating Token Index: 522

Text #11

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7808. Min Act: -0.0000

Data Index: 474495 (The Pile)

Max Activating Token Index: 529

Text #12

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.8243. Min Act: -0.0000

Data Index: 1318924 (The Pile)

Max Activating Token Index: 677

Text #13

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7896. Min Act: -0.0000

Data Index: 1818474 (The Pile)

Max Activating Token Index: 788

Text #14

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7574. Min Act: -0.0000

Data Index: 852773 (The Pile)

Max Activating Token Index: 821

Text #15

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7012. Min Act: -0.0000

Data Index: 555458 (The Pile)

Max Activating Token Index: 729

Text #16

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7428. Min Act: -0.0000

Data Index: 1823377 (The Pile)

Max Activating Token Index: 818

Text #17

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7303. Min Act: -0.0000

Data Index: 1170888 (The Pile)

Max Activating Token Index: 955

Text #18

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7268. Min Act: -0.0000

Data Index: 708842 (The Pile)

Max Activating Token Index: 840

Text #19

Max Range: 1.1978. Min Range: -1.1978

Max Act: 0.7535. Min Act: -0.0000

Data Index: 877363 (The Pile)

Max Activating Token Index: 972