Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1113 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.5836. Min Act: -0.0000

Data Index: 970793 (The Pile)

Max Activating Token Index: 537

Text #1

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.8650. Min Act: -0.0000

Data Index: 769385 (The Pile)

Max Activating Token Index: 68

Text #2

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3509. Min Act: -0.0000

Data Index: 1912454 (The Pile)

Max Activating Token Index: 119

Text #3

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.4982. Min Act: -0.0000

Data Index: 1630528 (The Pile)

Max Activating Token Index: 642

Text #4

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.4475. Min Act: -0.0000

Data Index: 1601940 (The Pile)

Max Activating Token Index: 848

Text #5

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.5581. Min Act: -0.0000

Data Index: 1530599 (The Pile)

Max Activating Token Index: 47

Text #6

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3619. Min Act: -0.0000

Data Index: 1746362 (The Pile)

Max Activating Token Index: 918

Text #7

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3637. Min Act: -0.0000

Data Index: 252430 (The Pile)

Max Activating Token Index: 658

Text #8

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3509. Min Act: -0.0000

Data Index: 142568 (The Pile)

Max Activating Token Index: 498

Text #9

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3441. Min Act: -0.0000

Data Index: 63387 (The Pile)

Max Activating Token Index: 646

Text #10

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.1824. Min Act: -0.0000

Data Index: 635833 (The Pile)

Max Activating Token Index: 969

Text #11

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.1641. Min Act: -0.0000

Data Index: 751273 (The Pile)

Max Activating Token Index: 728

Text #12

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3188. Min Act: -0.0000

Data Index: 1244361 (The Pile)

Max Activating Token Index: 509

Text #13

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.2690. Min Act: -0.0000

Data Index: 442975 (The Pile)

Max Activating Token Index: 505

Text #14

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.0807. Min Act: -0.0000

Data Index: 728971 (The Pile)

Max Activating Token Index: 412

Text #15

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3528. Min Act: -0.0000

Data Index: 1860169 (The Pile)

Max Activating Token Index: 80

Text #16

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.1943. Min Act: -0.0000

Data Index: 703683 (The Pile)

Max Activating Token Index: 120

Text #17

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.3291. Min Act: -0.0000

Data Index: 1196121 (The Pile)

Max Activating Token Index: 443

Text #18

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.0230. Min Act: -0.0000

Data Index: 1500807 (The Pile)

Max Activating Token Index: 203

Text #19

Max Range: 3.8650. Min Range: -3.8650

Max Act: 3.1581. Min Act: -0.0000

Data Index: 1913344 (The Pile)

Max Activating Token Index: 755