Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2023 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2796. Min Act: -0.0000

Data Index: 1205187 (The Pile)

Max Activating Token Index: 923

Text #1

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.3034. Min Act: -0.0000

Data Index: 1688616 (The Pile)

Max Activating Token Index: 1002

Text #2

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2607. Min Act: -0.0000

Data Index: 1710118 (The Pile)

Max Activating Token Index: 583

Text #3

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2435. Min Act: -0.0000

Data Index: 768205 (The Pile)

Max Activating Token Index: 893

Text #4

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2250. Min Act: -0.0000

Data Index: 860155 (The Pile)

Max Activating Token Index: 709

Text #5

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2313. Min Act: -0.0000

Data Index: 1777676 (The Pile)

Max Activating Token Index: 889

Text #6

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2314. Min Act: -0.0000

Data Index: 1850313 (The Pile)

Max Activating Token Index: 696

Text #7

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2198. Min Act: 0.0000

Data Index: 1878849 (The Pile)

Max Activating Token Index: 346

Text #8

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2270. Min Act: -0.0000

Data Index: 605662 (The Pile)

Max Activating Token Index: 813

Text #9

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.1994. Min Act: -0.0000

Data Index: 1508534 (The Pile)

Max Activating Token Index: 536

Text #10

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2100. Min Act: -0.0000

Data Index: 1029308 (The Pile)

Max Activating Token Index: 931

Text #11

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.1898. Min Act: -0.0000

Data Index: 746390 (The Pile)

Max Activating Token Index: 834

Text #12

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.1982. Min Act: -0.0000

Data Index: 1795285 (The Pile)

Max Activating Token Index: 979

Text #13

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.1970. Min Act: -0.0000

Data Index: 1218771 (The Pile)

Max Activating Token Index: 893

Text #14

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2090. Min Act: -0.0000

Data Index: 1748451 (The Pile)

Max Activating Token Index: 900

Text #15

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.1938. Min Act: -0.0000

Data Index: 1966774 (The Pile)

Max Activating Token Index: 706

Text #16

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2017. Min Act: -0.0000

Data Index: 1310590 (The Pile)

Max Activating Token Index: 964

Text #17

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.1895. Min Act: -0.0000

Data Index: 1222370 (The Pile)

Max Activating Token Index: 510

Text #18

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.2047. Min Act: -0.0000

Data Index: 1270156 (The Pile)

Max Activating Token Index: 872

Text #19

Max Range: 0.3034. Min Range: -0.3034

Max Act: 0.1923. Min Act: -0.0000

Data Index: 1599570 (The Pile)

Max Activating Token Index: 290