Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4424 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.4642. Min Act: 0.0000

Data Index: 1358280 (The Pile)

Max Activating Token Index: 331

Text #1

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.4353. Min Act: -0.0000

Data Index: 973877 (The Pile)

Max Activating Token Index: 816

Text #2

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.4290. Min Act: -0.0000

Data Index: 638044 (The Pile)

Max Activating Token Index: 485

Text #3

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.4130. Min Act: -0.0000

Data Index: 920750 (The Pile)

Max Activating Token Index: 511

Text #4

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3994. Min Act: -0.0000

Data Index: 1347048 (The Pile)

Max Activating Token Index: 725

Text #5

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3984. Min Act: -0.0000

Data Index: 1179866 (The Pile)

Max Activating Token Index: 357

Text #6

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.4092. Min Act: -0.0000

Data Index: 368479 (The Pile)

Max Activating Token Index: 400

Text #7

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3616. Min Act: -0.0000

Data Index: 1474283 (The Pile)

Max Activating Token Index: 226

Text #8

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3543. Min Act: -0.0000

Data Index: 1841499 (The Pile)

Max Activating Token Index: 552

Text #9

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3915. Min Act: -0.0000

Data Index: 905846 (The Pile)

Max Activating Token Index: 491

Text #10

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3783. Min Act: -0.0000

Data Index: 622201 (The Pile)

Max Activating Token Index: 106

Text #11

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3653. Min Act: -0.0000

Data Index: 1139347 (The Pile)

Max Activating Token Index: 592

Text #12

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3626. Min Act: -0.0000

Data Index: 207105 (The Pile)

Max Activating Token Index: 546

Text #13

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3572. Min Act: -0.0000

Data Index: 119573 (The Pile)

Max Activating Token Index: 487

Text #14

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.4265. Min Act: -0.0000

Data Index: 1209395 (The Pile)

Max Activating Token Index: 828

Text #15

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3572. Min Act: -0.0000

Data Index: 1676682 (The Pile)

Max Activating Token Index: 82

Text #16

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3422. Min Act: -0.0000

Data Index: 1645108 (The Pile)

Max Activating Token Index: 445

Text #17

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3296. Min Act: -0.0000

Data Index: 655745 (The Pile)

Max Activating Token Index: 431

Text #18

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3238. Min Act: -0.0000

Data Index: 1829535 (The Pile)

Max Activating Token Index: 598

Text #19

Max Range: 0.4642. Min Range: -0.4642

Max Act: 0.3182. Min Act: -0.0000

Data Index: 917817 (The Pile)

Max Activating Token Index: 592