Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2250 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.4305. Min Act: -0.0000

Data Index: 134035 (The Pile)

Max Activating Token Index: 436

Text #1

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.4645. Min Act: -0.0000

Data Index: 628269 (The Pile)

Max Activating Token Index: 191

Text #2

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.4210. Min Act: -0.0000

Data Index: 623779 (The Pile)

Max Activating Token Index: 447

Text #3

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.4134. Min Act: -0.0000

Data Index: 1883124 (The Pile)

Max Activating Token Index: 466

Text #4

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3988. Min Act: -0.0000

Data Index: 1053324 (The Pile)

Max Activating Token Index: 389

Text #5

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3931. Min Act: -0.0000

Data Index: 1011784 (The Pile)

Max Activating Token Index: 789

Text #6

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3902. Min Act: -0.0000

Data Index: 658378 (The Pile)

Max Activating Token Index: 676

Text #7

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3905. Min Act: -0.0000

Data Index: 1114029 (The Pile)

Max Activating Token Index: 713

Text #8

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3941. Min Act: -0.0000

Data Index: 1064483 (The Pile)

Max Activating Token Index: 423

Text #9

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3963. Min Act: -0.0000

Data Index: 1800124 (The Pile)

Max Activating Token Index: 586

Text #10

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3949. Min Act: -0.0000

Data Index: 309369 (The Pile)

Max Activating Token Index: 413

Text #11

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.4052. Min Act: -0.0000

Data Index: 1234564 (The Pile)

Max Activating Token Index: 996

Text #12

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3839. Min Act: -0.0000

Data Index: 356701 (The Pile)

Max Activating Token Index: 774

Text #13

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3781. Min Act: -0.0000

Data Index: 1327424 (The Pile)

Max Activating Token Index: 178

Text #14

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3895. Min Act: -0.0000

Data Index: 1931474 (The Pile)

Max Activating Token Index: 875

Text #15

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3805. Min Act: -0.0000

Data Index: 759082 (The Pile)

Max Activating Token Index: 959

Text #16

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3863. Min Act: -0.0000

Data Index: 1233673 (The Pile)

Max Activating Token Index: 208

Text #17

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3538. Min Act: -0.0000

Data Index: 1825027 (The Pile)

Max Activating Token Index: 846

Text #18

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3759. Min Act: -0.0000

Data Index: 700347 (The Pile)

Max Activating Token Index: 818

Text #19

Max Range: 0.4645. Min Range: -0.4645

Max Act: 0.3613. Min Act: -0.0000

Data Index: 1382541 (The Pile)

Max Activating Token Index: 500