Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3548 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.3701. Min Range: -4.3701

Max Act: 4.3521. Min Act: -0.0000

Data Index: 666794 (The Pile)

Max Activating Token Index: 828

Text #1

Max Range: 4.3701. Min Range: -4.3701

Max Act: 4.3701. Min Act: -0.0000

Data Index: 1374639 (The Pile)

Max Activating Token Index: 517

Text #2

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.9162. Min Act: -0.0000

Data Index: 563486 (The Pile)

Max Activating Token Index: 691

Text #3

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.8963. Min Act: -0.0000

Data Index: 927730 (The Pile)

Max Activating Token Index: 896

Text #4

Max Range: 4.3701. Min Range: -4.3701

Max Act: 4.0053. Min Act: -0.0000

Data Index: 1961173 (The Pile)

Max Activating Token Index: 213

Text #5

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.9063. Min Act: -0.0000

Data Index: 1991311 (The Pile)

Max Activating Token Index: 443

Text #6

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.8944. Min Act: -0.0000

Data Index: 950605 (The Pile)

Max Activating Token Index: 111

Text #7

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.8702. Min Act: -0.0000

Data Index: 1458174 (The Pile)

Max Activating Token Index: 452

Text #8

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.8523. Min Act: -0.0000

Data Index: 1052481 (The Pile)

Max Activating Token Index: 473

Text #9

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.6404. Min Act: -0.0000

Data Index: 484810 (The Pile)

Max Activating Token Index: 365

Text #10

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.7282. Min Act: -0.0000

Data Index: 1049320 (The Pile)

Max Activating Token Index: 623

Text #11

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.7193. Min Act: -0.0000

Data Index: 111717 (The Pile)

Max Activating Token Index: 784

Text #12

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.5488. Min Act: -0.0000

Data Index: 1463331 (The Pile)

Max Activating Token Index: 715

Text #13

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.7140. Min Act: -0.0000

Data Index: 533133 (The Pile)

Max Activating Token Index: 645

Text #14

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.3903. Min Act: -0.0000

Data Index: 616149 (The Pile)

Max Activating Token Index: 108

Text #15

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.8558. Min Act: -0.0000

Data Index: 1720636 (The Pile)

Max Activating Token Index: 126

Text #16

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.2257. Min Act: -0.0000

Data Index: 117073 (The Pile)

Max Activating Token Index: 947

Text #17

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.6917. Min Act: -0.0000

Data Index: 1491866 (The Pile)

Max Activating Token Index: 103

Text #18

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.8469. Min Act: -0.0000

Data Index: 1856219 (The Pile)

Max Activating Token Index: 532

Text #19

Max Range: 4.3701. Min Range: -4.3701

Max Act: 3.6675. Min Act: -0.0000

Data Index: 1471217 (The Pile)

Max Activating Token Index: 895