Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4440 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1593. Min Act: -0.0000

Data Index: 319575 (The Pile)

Max Activating Token Index: 226

Text #1

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1331. Min Act: -0.0000

Data Index: 155018 (The Pile)

Max Activating Token Index: 970

Text #2

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1392. Min Act: -0.0000

Data Index: 65543 (The Pile)

Max Activating Token Index: 44

Text #3

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1250. Min Act: -0.0000

Data Index: 1138673 (The Pile)

Max Activating Token Index: 52

Text #4

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1308. Min Act: -0.0000

Data Index: 810617 (The Pile)

Max Activating Token Index: 1008

Text #5

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1272. Min Act: -0.0000

Data Index: 34772 (The Pile)

Max Activating Token Index: 90

Text #6

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1581. Min Act: -0.0000

Data Index: 1936122 (The Pile)

Max Activating Token Index: 1021

Text #7

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1089. Min Act: -0.0000

Data Index: 1760818 (The Pile)

Max Activating Token Index: 80

Text #8

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1186. Min Act: -0.0000

Data Index: 258252 (The Pile)

Max Activating Token Index: 306

Text #9

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1208. Min Act: -0.0000

Data Index: 507733 (The Pile)

Max Activating Token Index: 241

Text #10

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1025. Min Act: -0.0000

Data Index: 367933 (The Pile)

Max Activating Token Index: 972

Text #11

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.0931. Min Act: -0.0000

Data Index: 1669186 (The Pile)

Max Activating Token Index: 1022

Text #12

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.0840. Min Act: -0.0000

Data Index: 1194216 (The Pile)

Max Activating Token Index: 990

Text #13

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1012. Min Act: -0.0000

Data Index: 1299854 (The Pile)

Max Activating Token Index: 110

Text #14

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1106. Min Act: -0.0000

Data Index: 1032944 (The Pile)

Max Activating Token Index: 192

Text #15

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.0960. Min Act: -0.0000

Data Index: 315787 (The Pile)

Max Activating Token Index: 49

Text #16

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1028. Min Act: -0.0000

Data Index: 1937581 (The Pile)

Max Activating Token Index: 140

Text #17

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.0897. Min Act: -0.0000

Data Index: 1884971 (The Pile)

Max Activating Token Index: 970

Text #18

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1032. Min Act: -0.0000

Data Index: 1667120 (The Pile)

Max Activating Token Index: 38

Text #19

Max Range: 0.1593. Min Range: -0.1593

Max Act: 0.1050. Min Act: -0.0000

Data Index: 1263384 (The Pile)

Max Activating Token Index: 235