Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1915 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.7945. Min Act: -0.0000

Data Index: 1196665 (The Pile)

Max Activating Token Index: 112

Text #1

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.7772. Min Act: -0.0000

Data Index: 542549 (The Pile)

Max Activating Token Index: 776

Text #2

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.6230. Min Act: -0.0000

Data Index: 875649 (The Pile)

Max Activating Token Index: 50

Text #3

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.6870. Min Act: -0.0000

Data Index: 530548 (The Pile)

Max Activating Token Index: 390

Text #4

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5370. Min Act: -0.0000

Data Index: 788453 (The Pile)

Max Activating Token Index: 988

Text #5

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.6338. Min Act: -0.0000

Data Index: 416819 (The Pile)

Max Activating Token Index: 322

Text #6

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.6009. Min Act: -0.0000

Data Index: 208225 (The Pile)

Max Activating Token Index: 901

Text #7

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.6563. Min Act: -0.0000

Data Index: 44279 (The Pile)

Max Activating Token Index: 775

Text #8

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5618. Min Act: -0.0000

Data Index: 1847188 (The Pile)

Max Activating Token Index: 658

Text #9

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5641. Min Act: -0.0000

Data Index: 1543805 (The Pile)

Max Activating Token Index: 438

Text #10

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.6352. Min Act: -0.0000

Data Index: 1364058 (The Pile)

Max Activating Token Index: 641

Text #11

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5869. Min Act: -0.0000

Data Index: 1820189 (The Pile)

Max Activating Token Index: 592

Text #12

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5460. Min Act: -0.0000

Data Index: 395738 (The Pile)

Max Activating Token Index: 666

Text #13

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5615. Min Act: -0.0000

Data Index: 962219 (The Pile)

Max Activating Token Index: 1008

Text #14

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5334. Min Act: -0.0000

Data Index: 539821 (The Pile)

Max Activating Token Index: 509

Text #15

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5390. Min Act: -0.0000

Data Index: 810083 (The Pile)

Max Activating Token Index: 350

Text #16

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5317. Min Act: -0.0000

Data Index: 1900987 (The Pile)

Max Activating Token Index: 230

Text #17

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5289. Min Act: -0.0000

Data Index: 1752680 (The Pile)

Max Activating Token Index: 112

Text #18

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5388. Min Act: -0.0000

Data Index: 1227339 (The Pile)

Max Activating Token Index: 725

Text #19

Max Range: 0.7945. Min Range: -0.7945

Max Act: 0.5208. Min Act: -0.0000

Data Index: 386300 (The Pile)

Max Activating Token Index: 328