Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4571 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.8865. Min Act: -0.0000

Data Index: 966703 (The Pile)

Max Activating Token Index: 576

Text #1

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.8084. Min Act: -0.0000

Data Index: 346238 (The Pile)

Max Activating Token Index: 152

Text #2

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.7175. Min Act: -0.0000

Data Index: 1832691 (The Pile)

Max Activating Token Index: 355

Text #3

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.8321. Min Act: -0.0000

Data Index: 1407349 (The Pile)

Max Activating Token Index: 197

Text #4

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.7468. Min Act: -0.0000

Data Index: 1925140 (The Pile)

Max Activating Token Index: 348

Text #5

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6717. Min Act: -0.0000

Data Index: 970975 (The Pile)

Max Activating Token Index: 63

Text #6

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.7351. Min Act: -0.0000

Data Index: 770142 (The Pile)

Max Activating Token Index: 204

Text #7

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6886. Min Act: -0.0000

Data Index: 405103 (The Pile)

Max Activating Token Index: 382

Text #8

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6993. Min Act: -0.0000

Data Index: 682112 (The Pile)

Max Activating Token Index: 413

Text #9

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.5978. Min Act: -0.0000

Data Index: 828442 (The Pile)

Max Activating Token Index: 41

Text #10

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6117. Min Act: -0.0000

Data Index: 1481747 (The Pile)

Max Activating Token Index: 514

Text #11

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6474. Min Act: -0.0000

Data Index: 1200764 (The Pile)

Max Activating Token Index: 443

Text #12

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6444. Min Act: -0.0000

Data Index: 450878 (The Pile)

Max Activating Token Index: 337

Text #13

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6207. Min Act: -0.0000

Data Index: 1283260 (The Pile)

Max Activating Token Index: 388

Text #14

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6126. Min Act: -0.0000

Data Index: 811628 (The Pile)

Max Activating Token Index: 442

Text #15

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6715. Min Act: -0.0000

Data Index: 573463 (The Pile)

Max Activating Token Index: 857

Text #16

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6273. Min Act: -0.0000

Data Index: 437829 (The Pile)

Max Activating Token Index: 153

Text #17

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.5970. Min Act: -0.0000

Data Index: 379948 (The Pile)

Max Activating Token Index: 440

Text #18

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6324. Min Act: -0.0000

Data Index: 1334562 (The Pile)

Max Activating Token Index: 479

Text #19

Max Range: 0.8865. Min Range: -0.8865

Max Act: 0.6308. Min Act: -0.0000

Data Index: 212847 (The Pile)

Max Activating Token Index: 1022