Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3746 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.0892. Min Act: -0.0000

Data Index: 1003090 (The Pile)

Max Activating Token Index: 884

Text #1

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.4756. Min Act: -0.0000

Data Index: 387091 (The Pile)

Max Activating Token Index: 381

Text #2

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.6199. Min Act: -0.0000

Data Index: 1162470 (The Pile)

Max Activating Token Index: 476

Text #3

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.3204. Min Act: -0.0000

Data Index: 1078143 (The Pile)

Max Activating Token Index: 470

Text #4

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.1619. Min Act: -0.0000

Data Index: 817988 (The Pile)

Max Activating Token Index: 766

Text #5

Max Range: 3.6199. Min Range: -3.6199

Max Act: 2.9004. Min Act: -0.0000

Data Index: 480850 (The Pile)

Max Activating Token Index: 703

Text #6

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.0822. Min Act: -0.0000

Data Index: 1582673 (The Pile)

Max Activating Token Index: 658

Text #7

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.1743. Min Act: -0.0000

Data Index: 1261475 (The Pile)

Max Activating Token Index: 574

Text #8

Max Range: 3.6199. Min Range: -3.6199

Max Act: 2.9663. Min Act: -0.0000

Data Index: 124799 (The Pile)

Max Activating Token Index: 832

Text #9

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.1390. Min Act: -0.0000

Data Index: 1954611 (The Pile)

Max Activating Token Index: 244

Text #10

Max Range: 3.6199. Min Range: -3.6199

Max Act: 2.9857. Min Act: -0.0000

Data Index: 1093193 (The Pile)

Max Activating Token Index: 363

Text #11

Max Range: 3.6199. Min Range: -3.6199

Max Act: 2.7879. Min Act: -0.0000

Data Index: 1863229 (The Pile)

Max Activating Token Index: 561

Text #12

Max Range: 3.6199. Min Range: -3.6199

Max Act: 2.9255. Min Act: -0.0000

Data Index: 1342157 (The Pile)

Max Activating Token Index: 231

Text #13

Max Range: 3.6199. Min Range: -3.6199

Max Act: 2.9779. Min Act: -0.0000

Data Index: 1634217 (The Pile)

Max Activating Token Index: 437

Text #14

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.0658. Min Act: -0.0000

Data Index: 1238415 (The Pile)

Max Activating Token Index: 110

Text #15

Max Range: 3.6199. Min Range: -3.6199

Max Act: 2.9229. Min Act: -0.0000

Data Index: 333718 (The Pile)

Max Activating Token Index: 118

Text #16

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.1493. Min Act: -0.0000

Data Index: 1661200 (The Pile)

Max Activating Token Index: 241

Text #17

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.0275. Min Act: -0.0000

Data Index: 1457887 (The Pile)

Max Activating Token Index: 542

Text #18

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.1834. Min Act: -0.0000

Data Index: 219749 (The Pile)

Max Activating Token Index: 98

Text #19

Max Range: 3.6199. Min Range: -3.6199

Max Act: 3.0306. Min Act: -0.0000

Data Index: 115569 (The Pile)

Max Activating Token Index: 619