Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2936 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.5403. Min Range: -4.5403

Max Act: 4.5403. Min Act: -0.0000

Data Index: 1814861 (The Pile)

Max Activating Token Index: 378

Text #1

Max Range: 4.5403. Min Range: -4.5403

Max Act: 4.3227. Min Act: -0.0000

Data Index: 858288 (The Pile)

Max Activating Token Index: 192

Text #2

Max Range: 4.5403. Min Range: -4.5403

Max Act: 4.2178. Min Act: -0.0000

Data Index: 1847307 (The Pile)

Max Activating Token Index: 275

Text #3

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.7923. Min Act: -0.0000

Data Index: 1713796 (The Pile)

Max Activating Token Index: 648

Text #4

Max Range: 4.5403. Min Range: -4.5403

Max Act: 4.3738. Min Act: -0.0000

Data Index: 71801 (The Pile)

Max Activating Token Index: 257

Text #5

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.9875. Min Act: -0.0000

Data Index: 1070317 (The Pile)

Max Activating Token Index: 577

Text #6

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.8350. Min Act: -0.0000

Data Index: 379644 (The Pile)

Max Activating Token Index: 691

Text #7

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.8014. Min Act: -0.0000

Data Index: 432559 (The Pile)

Max Activating Token Index: 447

Text #8

Max Range: 4.5403. Min Range: -4.5403

Max Act: 4.0022. Min Act: -0.0000

Data Index: 1407073 (The Pile)

Max Activating Token Index: 455

Text #9

Max Range: 4.5403. Min Range: -4.5403

Max Act: 4.1189. Min Act: -0.0000

Data Index: 730149 (The Pile)

Max Activating Token Index: 521

Text #10

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.8642. Min Act: -0.0000

Data Index: 693633 (The Pile)

Max Activating Token Index: 549

Text #11

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.5669. Min Act: -0.0000

Data Index: 463772 (The Pile)

Max Activating Token Index: 79

Text #12

Max Range: 4.5403. Min Range: -4.5403

Max Act: 4.2132. Min Act: -0.0000

Data Index: 463693 (The Pile)

Max Activating Token Index: 130

Text #13

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.6991. Min Act: -0.0000

Data Index: 1285197 (The Pile)

Max Activating Token Index: 471

Text #14

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.4181. Min Act: -0.0000

Data Index: 1605575 (The Pile)

Max Activating Token Index: 449

Text #15

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.7396. Min Act: -0.0000

Data Index: 406736 (The Pile)

Max Activating Token Index: 508

Text #16

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.1391. Min Act: -0.0000

Data Index: 501148 (The Pile)

Max Activating Token Index: 701

Text #17

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.2654. Min Act: -0.0000

Data Index: 850243 (The Pile)

Max Activating Token Index: 236

Text #18

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.2393. Min Act: -0.0000

Data Index: 1022007 (The Pile)

Max Activating Token Index: 636

Text #19

Max Range: 4.5403. Min Range: -4.5403

Max Act: 3.4595. Min Act: -0.0000

Data Index: 1658764 (The Pile)

Max Activating Token Index: 111