Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3569 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.3251. Min Range: -3.3251

Max Act: 3.3251. Min Act: -0.0000

Data Index: 852391 (The Pile)

Max Activating Token Index: 174

Text #1

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.5051. Min Act: -0.0000

Data Index: 1327005 (The Pile)

Max Activating Token Index: 752

Text #2

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.3697. Min Act: -0.0000

Data Index: 1642081 (The Pile)

Max Activating Token Index: 129

Text #3

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.4178. Min Act: -0.0000

Data Index: 38335 (The Pile)

Max Activating Token Index: 309

Text #4

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.4320. Min Act: -0.0000

Data Index: 1776124 (The Pile)

Max Activating Token Index: 109

Text #5

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.2161. Min Act: -0.0000

Data Index: 431345 (The Pile)

Max Activating Token Index: 601

Text #6

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.1436. Min Act: -0.0000

Data Index: 876936 (The Pile)

Max Activating Token Index: 634

Text #7

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.8138. Min Act: -0.0000

Data Index: 1564746 (The Pile)

Max Activating Token Index: 72

Text #8

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.3485. Min Act: -0.0000

Data Index: 1123096 (The Pile)

Max Activating Token Index: 167

Text #9

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.0858. Min Act: -0.0000

Data Index: 35811 (The Pile)

Max Activating Token Index: 530

Text #10

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.1361. Min Act: -0.0000

Data Index: 1439464 (The Pile)

Max Activating Token Index: 578

Text #11

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.7978. Min Act: -0.0000

Data Index: 376548 (The Pile)

Max Activating Token Index: 873

Text #12

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.6943. Min Act: -0.0000

Data Index: 1647187 (The Pile)

Max Activating Token Index: 864

Text #13

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.9889. Min Act: -0.0000

Data Index: 1349659 (The Pile)

Max Activating Token Index: 219

Text #14

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.7025. Min Act: -0.0000

Data Index: 663741 (The Pile)

Max Activating Token Index: 897

Text #15

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.0549. Min Act: -0.0000

Data Index: 1273406 (The Pile)

Max Activating Token Index: 174

Text #16

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.6476. Min Act: -0.0000

Data Index: 467594 (The Pile)

Max Activating Token Index: 957

Text #17

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.8168. Min Act: -0.0000

Data Index: 1906965 (The Pile)

Max Activating Token Index: 694

Text #18

Max Range: 3.3251. Min Range: -3.3251

Max Act: 1.7939. Min Act: -0.0000

Data Index: 199903 (The Pile)

Max Activating Token Index: 543

Text #19

Max Range: 3.3251. Min Range: -3.3251

Max Act: 2.0342. Min Act: -0.0000

Data Index: 1317563 (The Pile)

Max Activating Token Index: 902