Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2768 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.7629. Min Act: -0.0000

Data Index: 1364439 (The Pile)

Max Activating Token Index: 598

Text #1

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6870. Min Act: -0.0000

Data Index: 1216719 (The Pile)

Max Activating Token Index: 1009

Text #2

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6842. Min Act: -0.0000

Data Index: 939033 (The Pile)

Max Activating Token Index: 478

Text #3

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6612. Min Act: -0.0000

Data Index: 1097033 (The Pile)

Max Activating Token Index: 364

Text #4

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6515. Min Act: -0.0000

Data Index: 875753 (The Pile)

Max Activating Token Index: 629

Text #5

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6338. Min Act: -0.0000

Data Index: 783448 (The Pile)

Max Activating Token Index: 649

Text #6

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.5889. Min Act: -0.0000

Data Index: 669484 (The Pile)

Max Activating Token Index: 25

Text #7

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6488. Min Act: -0.0000

Data Index: 299401 (The Pile)

Max Activating Token Index: 617

Text #8

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6845. Min Act: -0.0000

Data Index: 984228 (The Pile)

Max Activating Token Index: 994

Text #9

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6310. Min Act: -0.0000

Data Index: 1158262 (The Pile)

Max Activating Token Index: 39

Text #10

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.5954. Min Act: -0.0000

Data Index: 1518886 (The Pile)

Max Activating Token Index: 821

Text #11

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6146. Min Act: -0.0000

Data Index: 513211 (The Pile)

Max Activating Token Index: 1021

Text #12

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6302. Min Act: -0.0000

Data Index: 1634633 (The Pile)

Max Activating Token Index: 592

Text #13

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6246. Min Act: -0.0000

Data Index: 406895 (The Pile)

Max Activating Token Index: 279

Text #14

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6135. Min Act: -0.0000

Data Index: 119385 (The Pile)

Max Activating Token Index: 624

Text #15

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6575. Min Act: -0.0000

Data Index: 1137487 (The Pile)

Max Activating Token Index: 156

Text #16

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6008. Min Act: -0.0000

Data Index: 1149764 (The Pile)

Max Activating Token Index: 987

Text #17

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.6012. Min Act: -0.0000

Data Index: 1174860 (The Pile)

Max Activating Token Index: 558

Text #18

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.5998. Min Act: -0.0000

Data Index: 1149485 (The Pile)

Max Activating Token Index: 315

Text #19

Max Range: 0.7629. Min Range: -0.7629

Max Act: 0.5782. Min Act: -0.0000

Data Index: 1037052 (The Pile)

Max Activating Token Index: 853