Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4014 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.6576. Min Act: -0.0000

Data Index: 937552 (The Pile)

Max Activating Token Index: 486

Text #1

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.9071. Min Act: -0.0000

Data Index: 759365 (The Pile)

Max Activating Token Index: 311

Text #2

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.9705. Min Act: -0.0000

Data Index: 974411 (The Pile)

Max Activating Token Index: 660

Text #3

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.6695. Min Act: -0.0000

Data Index: 756909 (The Pile)

Max Activating Token Index: 144

Text #4

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.5030. Min Act: -0.0000

Data Index: 1449126 (The Pile)

Max Activating Token Index: 510

Text #5

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.4522. Min Act: -0.0000

Data Index: 414418 (The Pile)

Max Activating Token Index: 593

Text #6

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.3173. Min Act: -0.0000

Data Index: 277729 (The Pile)

Max Activating Token Index: 284

Text #7

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.3429. Min Act: -0.0000

Data Index: 218073 (The Pile)

Max Activating Token Index: 491

Text #8

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.1686. Min Act: -0.0000

Data Index: 373499 (The Pile)

Max Activating Token Index: 293

Text #9

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.2744. Min Act: -0.0000

Data Index: 22324 (The Pile)

Max Activating Token Index: 495

Text #10

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.1273. Min Act: -0.0000

Data Index: 880171 (The Pile)

Max Activating Token Index: 938

Text #11

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.2511. Min Act: -0.0000

Data Index: 1089434 (The Pile)

Max Activating Token Index: 518

Text #12

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.1111. Min Act: -0.0000

Data Index: 918033 (The Pile)

Max Activating Token Index: 404

Text #13

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.2307. Min Act: -0.0000

Data Index: 1361060 (The Pile)

Max Activating Token Index: 831

Text #14

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.0943. Min Act: -0.0000

Data Index: 1284793 (The Pile)

Max Activating Token Index: 985

Text #15

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.2955. Min Act: -0.0000

Data Index: 1243373 (The Pile)

Max Activating Token Index: 353

Text #16

Max Range: 3.6576. Min Range: -3.6576

Max Act: 1.9458. Min Act: -0.0000

Data Index: 765880 (The Pile)

Max Activating Token Index: 359

Text #17

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.5634. Min Act: -0.0000

Data Index: 487305 (The Pile)

Max Activating Token Index: 78

Text #18

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.1482. Min Act: -0.0000

Data Index: 392330 (The Pile)

Max Activating Token Index: 571

Text #19

Max Range: 3.6576. Min Range: -3.6576

Max Act: 2.1480. Min Act: -0.0000

Data Index: 717561 (The Pile)

Max Activating Token Index: 954