Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4427 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1037. Min Range: -1.1037

Max Act: 1.1037. Min Act: -0.0000

Data Index: 751988 (The Pile)

Max Activating Token Index: 240

Text #1

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.9007. Min Act: -0.0000

Data Index: 851949 (The Pile)

Max Activating Token Index: 79

Text #2

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.8649. Min Act: -0.0000

Data Index: 1563769 (The Pile)

Max Activating Token Index: 77

Text #3

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.8506. Min Act: -0.0000

Data Index: 1471930 (The Pile)

Max Activating Token Index: 100

Text #4

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7878. Min Act: -0.0000

Data Index: 336783 (The Pile)

Max Activating Token Index: 703

Text #5

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7480. Min Act: -0.0000

Data Index: 179986 (The Pile)

Max Activating Token Index: 254

Text #6

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7875. Min Act: -0.0000

Data Index: 279999 (The Pile)

Max Activating Token Index: 643

Text #7

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7748. Min Act: -0.0000

Data Index: 68583 (The Pile)

Max Activating Token Index: 240

Text #8

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.6960. Min Act: -0.0000

Data Index: 1756392 (The Pile)

Max Activating Token Index: 127

Text #9

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7819. Min Act: -0.0000

Data Index: 1235367 (The Pile)

Max Activating Token Index: 903

Text #10

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7079. Min Act: -0.0000

Data Index: 21557 (The Pile)

Max Activating Token Index: 148

Text #11

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7021. Min Act: -0.0000

Data Index: 1360820 (The Pile)

Max Activating Token Index: 763

Text #12

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7383. Min Act: -0.0000

Data Index: 593589 (The Pile)

Max Activating Token Index: 911

Text #13

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7250. Min Act: -0.0000

Data Index: 418774 (The Pile)

Max Activating Token Index: 472

Text #14

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7190. Min Act: -0.0000

Data Index: 1719425 (The Pile)

Max Activating Token Index: 409

Text #15

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.6947. Min Act: -0.0000

Data Index: 1006554 (The Pile)

Max Activating Token Index: 666

Text #16

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7163. Min Act: -0.0000

Data Index: 348898 (The Pile)

Max Activating Token Index: 947

Text #17

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7402. Min Act: -0.0000

Data Index: 1748752 (The Pile)

Max Activating Token Index: 64

Text #18

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.6411. Min Act: -0.0000

Data Index: 1260266 (The Pile)

Max Activating Token Index: 903

Text #19

Max Range: 1.1037. Min Range: -1.1037

Max Act: 0.7050. Min Act: -0.0000

Data Index: 982171 (The Pile)

Max Activating Token Index: 52