Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2110 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.7252. Min Act: -0.0000

Data Index: 577751 (The Pile)

Max Activating Token Index: 864

Text #1

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.7572. Min Act: -0.0000

Data Index: 1855448 (The Pile)

Max Activating Token Index: 137

Text #2

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.6786. Min Act: -0.0000

Data Index: 677860 (The Pile)

Max Activating Token Index: 678

Text #3

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.6836. Min Act: -0.0000

Data Index: 1206402 (The Pile)

Max Activating Token Index: 295

Text #4

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.7076. Min Act: -0.0000

Data Index: 1875434 (The Pile)

Max Activating Token Index: 672

Text #5

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.6055. Min Act: -0.0000

Data Index: 1395647 (The Pile)

Max Activating Token Index: 438

Text #6

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5933. Min Act: -0.0000

Data Index: 1434714 (The Pile)

Max Activating Token Index: 158

Text #7

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5885. Min Act: -0.0000

Data Index: 589667 (The Pile)

Max Activating Token Index: 748

Text #8

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5756. Min Act: -0.0000

Data Index: 903341 (The Pile)

Max Activating Token Index: 260

Text #9

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5695. Min Act: -0.0000

Data Index: 1680969 (The Pile)

Max Activating Token Index: 407

Text #10

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.6323. Min Act: -0.0000

Data Index: 852668 (The Pile)

Max Activating Token Index: 180

Text #11

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5326. Min Act: -0.0000

Data Index: 958395 (The Pile)

Max Activating Token Index: 650

Text #12

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5305. Min Act: -0.0000

Data Index: 1725741 (The Pile)

Max Activating Token Index: 371

Text #13

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5603. Min Act: -0.0000

Data Index: 1492285 (The Pile)

Max Activating Token Index: 80

Text #14

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5133. Min Act: -0.0000

Data Index: 617451 (The Pile)

Max Activating Token Index: 634

Text #15

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5515. Min Act: -0.0000

Data Index: 1247142 (The Pile)

Max Activating Token Index: 82

Text #16

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5370. Min Act: -0.0000

Data Index: 1767827 (The Pile)

Max Activating Token Index: 470

Text #17

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5118. Min Act: -0.0000

Data Index: 1534239 (The Pile)

Max Activating Token Index: 466

Text #18

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5124. Min Act: -0.0000

Data Index: 1985010 (The Pile)

Max Activating Token Index: 438

Text #19

Max Range: 0.7572. Min Range: -0.7572

Max Act: 0.5466. Min Act: -0.0000

Data Index: 651270 (The Pile)

Max Activating Token Index: 207