Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2638 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.5953. Min Act: -0.0000

Data Index: 688639 (The Pile)

Max Activating Token Index: 700

Text #1

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.5668. Min Act: -0.0000

Data Index: 435002 (The Pile)

Max Activating Token Index: 462

Text #2

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.4616. Min Act: -0.0000

Data Index: 210154 (The Pile)

Max Activating Token Index: 232

Text #3

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.4147. Min Act: -0.0000

Data Index: 652591 (The Pile)

Max Activating Token Index: 547

Text #4

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3446. Min Act: -0.0000

Data Index: 1007139 (The Pile)

Max Activating Token Index: 582

Text #5

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3513. Min Act: -0.0000

Data Index: 1290443 (The Pile)

Max Activating Token Index: 981

Text #6

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3588. Min Act: -0.0000

Data Index: 1548800 (The Pile)

Max Activating Token Index: 239

Text #7

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3533. Min Act: -0.0000

Data Index: 869693 (The Pile)

Max Activating Token Index: 693

Text #8

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3450. Min Act: -0.0000

Data Index: 344042 (The Pile)

Max Activating Token Index: 941

Text #9

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3520. Min Act: -0.0000

Data Index: 620580 (The Pile)

Max Activating Token Index: 932

Text #10

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3136. Min Act: -0.0000

Data Index: 263970 (The Pile)

Max Activating Token Index: 651

Text #11

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3355. Min Act: -0.0000

Data Index: 269312 (The Pile)

Max Activating Token Index: 656

Text #12

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3392. Min Act: -0.0000

Data Index: 363326 (The Pile)

Max Activating Token Index: 151

Text #13

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3069. Min Act: -0.0000

Data Index: 1506148 (The Pile)

Max Activating Token Index: 326

Text #14

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3189. Min Act: -0.0000

Data Index: 748082 (The Pile)

Max Activating Token Index: 309

Text #15

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3152. Min Act: -0.0000

Data Index: 613669 (The Pile)

Max Activating Token Index: 456

Text #16

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3024. Min Act: -0.0000

Data Index: 289662 (The Pile)

Max Activating Token Index: 844

Text #17

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.3049. Min Act: -0.0000

Data Index: 834475 (The Pile)

Max Activating Token Index: 142

Text #18

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.2903. Min Act: -0.0000

Data Index: 1025224 (The Pile)

Max Activating Token Index: 786

Text #19

Max Range: 0.5953. Min Range: -0.5953

Max Act: 0.2977. Min Act: -0.0000

Data Index: 1492924 (The Pile)

Max Activating Token Index: 917