Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1957 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.6050. Min Act: -0.0000

Data Index: 976495 (The Pile)

Max Activating Token Index: 960

Text #1

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.6093. Min Act: -0.0000

Data Index: 1820790 (The Pile)

Max Activating Token Index: 377

Text #2

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.5371. Min Act: -0.0000

Data Index: 741521 (The Pile)

Max Activating Token Index: 28

Text #3

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.5306. Min Act: -0.0000

Data Index: 940939 (The Pile)

Max Activating Token Index: 654

Text #4

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.5508. Min Act: -0.0000

Data Index: 108551 (The Pile)

Max Activating Token Index: 89

Text #5

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4581. Min Act: -0.0000

Data Index: 488980 (The Pile)

Max Activating Token Index: 446

Text #6

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.5162. Min Act: -0.0000

Data Index: 353271 (The Pile)

Max Activating Token Index: 337

Text #7

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4940. Min Act: -0.0000

Data Index: 1454699 (The Pile)

Max Activating Token Index: 615

Text #8

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4679. Min Act: -0.0000

Data Index: 1094937 (The Pile)

Max Activating Token Index: 733

Text #9

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.5060. Min Act: -0.0000

Data Index: 1266818 (The Pile)

Max Activating Token Index: 191

Text #10

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4392. Min Act: -0.0000

Data Index: 1769780 (The Pile)

Max Activating Token Index: 792

Text #11

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4834. Min Act: -0.0000

Data Index: 1240258 (The Pile)

Max Activating Token Index: 533

Text #12

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4807. Min Act: -0.0000

Data Index: 278687 (The Pile)

Max Activating Token Index: 971

Text #13

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4551. Min Act: -0.0000

Data Index: 541881 (The Pile)

Max Activating Token Index: 768

Text #14

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4776. Min Act: -0.0000

Data Index: 1222503 (The Pile)

Max Activating Token Index: 448

Text #15

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.5251. Min Act: -0.0000

Data Index: 277281 (The Pile)

Max Activating Token Index: 795

Text #16

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4886. Min Act: -0.0000

Data Index: 826616 (The Pile)

Max Activating Token Index: 459

Text #17

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4455. Min Act: -0.0000

Data Index: 1569825 (The Pile)

Max Activating Token Index: 244

Text #18

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4725. Min Act: -0.0000

Data Index: 663211 (The Pile)

Max Activating Token Index: 282

Text #19

Max Range: 0.6093. Min Range: -0.6093

Max Act: 0.4413. Min Act: -0.0000

Data Index: 510926 (The Pile)

Max Activating Token Index: 127