Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3039 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1781. Min Act: -0.0000

Data Index: 858574 (The Pile)

Max Activating Token Index: 615

Text #1

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1599. Min Act: -0.0000

Data Index: 1624939 (The Pile)

Max Activating Token Index: 329

Text #2

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1378. Min Act: -0.0000

Data Index: 622886 (The Pile)

Max Activating Token Index: 44

Text #3

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1247. Min Act: -0.0000

Data Index: 1126001 (The Pile)

Max Activating Token Index: 584

Text #4

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1237. Min Act: -0.0000

Data Index: 1493021 (The Pile)

Max Activating Token Index: 624

Text #5

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1254. Min Act: -0.0000

Data Index: 846251 (The Pile)

Max Activating Token Index: 546

Text #6

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1227. Min Act: -0.0000

Data Index: 943204 (The Pile)

Max Activating Token Index: 517

Text #7

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1118. Min Act: -0.0000

Data Index: 781510 (The Pile)

Max Activating Token Index: 426

Text #8

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1091. Min Act: -0.0000

Data Index: 1027598 (The Pile)

Max Activating Token Index: 240

Text #9

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1080. Min Act: -0.0000

Data Index: 1944804 (The Pile)

Max Activating Token Index: 60

Text #10

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1087. Min Act: -0.0000

Data Index: 385639 (The Pile)

Max Activating Token Index: 490

Text #11

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1054. Min Act: -0.0000

Data Index: 1591940 (The Pile)

Max Activating Token Index: 290

Text #12

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1051. Min Act: -0.0000

Data Index: 380059 (The Pile)

Max Activating Token Index: 903

Text #13

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.0844. Min Act: -0.0000

Data Index: 1734751 (The Pile)

Max Activating Token Index: 605

Text #14

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1092. Min Act: -0.0000

Data Index: 1826429 (The Pile)

Max Activating Token Index: 320

Text #15

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1094. Min Act: -0.0000

Data Index: 372038 (The Pile)

Max Activating Token Index: 215

Text #16

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1027. Min Act: -0.0000

Data Index: 1569034 (The Pile)

Max Activating Token Index: 280

Text #17

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.1009. Min Act: -0.0000

Data Index: 1653085 (The Pile)

Max Activating Token Index: 575

Text #18

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.0919. Min Act: -0.0000

Data Index: 327552 (The Pile)

Max Activating Token Index: 517

Text #19

Max Range: 0.1781. Min Range: -0.1781

Max Act: 0.0946. Min Act: -0.0000

Data Index: 238515 (The Pile)

Max Activating Token Index: 663