Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3019 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.5246. Min Act: -0.0000

Data Index: 1601755 (The Pile)

Max Activating Token Index: 930

Text #1

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.6721. Min Act: -0.0000

Data Index: 1399481 (The Pile)

Max Activating Token Index: 857

Text #2

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.6908. Min Act: -0.0000

Data Index: 344705 (The Pile)

Max Activating Token Index: 707

Text #3

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.4476. Min Act: -0.0000

Data Index: 718033 (The Pile)

Max Activating Token Index: 132

Text #4

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.4063. Min Act: -0.0000

Data Index: 20102 (The Pile)

Max Activating Token Index: 814

Text #5

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.3954. Min Act: 0.0000

Data Index: 1615121 (The Pile)

Max Activating Token Index: 217

Text #6

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.5559. Min Act: -0.0000

Data Index: 1622079 (The Pile)

Max Activating Token Index: 919

Text #7

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.1913. Min Act: -0.0000

Data Index: 137878 (The Pile)

Max Activating Token Index: 841

Text #8

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.3468. Min Act: -0.0000

Data Index: 1925980 (The Pile)

Max Activating Token Index: 59

Text #9

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.3838. Min Act: -0.0000

Data Index: 1928177 (The Pile)

Max Activating Token Index: 1000

Text #10

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.1436. Min Act: -0.0000

Data Index: 5934 (The Pile)

Max Activating Token Index: 841

Text #11

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.4696. Min Act: -0.0000

Data Index: 437237 (The Pile)

Max Activating Token Index: 363

Text #12

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.2235. Min Act: -0.0000

Data Index: 559272 (The Pile)

Max Activating Token Index: 173

Text #13

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.5619. Min Act: -0.0000

Data Index: 779245 (The Pile)

Max Activating Token Index: 719

Text #14

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.2249. Min Act: -0.0000

Data Index: 830257 (The Pile)

Max Activating Token Index: 998

Text #15

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.2101. Min Act: -0.0000

Data Index: 484399 (The Pile)

Max Activating Token Index: 110

Text #16

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.1574. Min Act: -0.0000

Data Index: 309801 (The Pile)

Max Activating Token Index: 887

Text #17

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.0513. Min Act: -0.0000

Data Index: 492513 (The Pile)

Max Activating Token Index: 799

Text #18

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.1135. Min Act: -0.0000

Data Index: 566970 (The Pile)

Max Activating Token Index: 935

Text #19

Max Range: 1.6908. Min Range: -1.6908

Max Act: 1.1097. Min Act: -0.0000

Data Index: 1885493 (The Pile)

Max Activating Token Index: 1008