Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3607 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.5410. Min Act: -0.0000

Data Index: 988662 (The Pile)

Max Activating Token Index: 458

Text #1

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.5409. Min Act: -0.0000

Data Index: 416617 (The Pile)

Max Activating Token Index: 479

Text #2

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.5182. Min Act: -0.0000

Data Index: 168307 (The Pile)

Max Activating Token Index: 616

Text #3

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.4905. Min Act: -0.0000

Data Index: 835305 (The Pile)

Max Activating Token Index: 303

Text #4

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.4641. Min Act: -0.0000

Data Index: 896135 (The Pile)

Max Activating Token Index: 730

Text #5

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.4674. Min Act: -0.0000

Data Index: 1159099 (The Pile)

Max Activating Token Index: 798

Text #6

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.4240. Min Act: -0.0000

Data Index: 1589677 (The Pile)

Max Activating Token Index: 811

Text #7

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.3808. Min Act: -0.0000

Data Index: 806664 (The Pile)

Max Activating Token Index: 327

Text #8

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.3341. Min Act: -0.0000

Data Index: 1985317 (The Pile)

Max Activating Token Index: 654

Text #9

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.3650. Min Act: -0.0000

Data Index: 1115766 (The Pile)

Max Activating Token Index: 101

Text #10

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.3111. Min Act: -0.0000

Data Index: 1540216 (The Pile)

Max Activating Token Index: 764

Text #11

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.3078. Min Act: -0.0000

Data Index: 1397193 (The Pile)

Max Activating Token Index: 657

Text #12

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2741. Min Act: -0.0000

Data Index: 746779 (The Pile)

Max Activating Token Index: 866

Text #13

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2844. Min Act: -0.0000

Data Index: 1783650 (The Pile)

Max Activating Token Index: 588

Text #14

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2729. Min Act: -0.0000

Data Index: 1617482 (The Pile)

Max Activating Token Index: 876

Text #15

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2710. Min Act: -0.0000

Data Index: 801287 (The Pile)

Max Activating Token Index: 388

Text #16

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2561. Min Act: -0.0000

Data Index: 1500424 (The Pile)

Max Activating Token Index: 353

Text #17

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2494. Min Act: -0.0000

Data Index: 1580905 (The Pile)

Max Activating Token Index: 630

Text #18

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2409. Min Act: -0.0000

Data Index: 1031700 (The Pile)

Max Activating Token Index: 1008

Text #19

Max Range: 0.5410. Min Range: -0.5410

Max Act: 0.2639. Min Act: -0.0000

Data Index: 1145123 (The Pile)

Max Activating Token Index: 793