Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2225 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.7876. Min Act: -0.0000

Data Index: 1148256 (The Pile)

Max Activating Token Index: 951

Text #1

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.8553. Min Act: -0.0000

Data Index: 1682148 (The Pile)

Max Activating Token Index: 502

Text #2

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.6775. Min Act: -0.0000

Data Index: 1892733 (The Pile)

Max Activating Token Index: 118

Text #3

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.5152. Min Act: -0.0000

Data Index: 17633 (The Pile)

Max Activating Token Index: 813

Text #4

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.7201. Min Act: -0.0000

Data Index: 1605017 (The Pile)

Max Activating Token Index: 168

Text #5

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.7280. Min Act: -0.0000

Data Index: 1591098 (The Pile)

Max Activating Token Index: 648

Text #6

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.5260. Min Act: -0.0000

Data Index: 1210871 (The Pile)

Max Activating Token Index: 791

Text #7

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.6424. Min Act: -0.0000

Data Index: 557695 (The Pile)

Max Activating Token Index: 524

Text #8

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.5376. Min Act: -0.0000

Data Index: 596889 (The Pile)

Max Activating Token Index: 15

Text #9

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.4227. Min Act: -0.0000

Data Index: 862158 (The Pile)

Max Activating Token Index: 369

Text #10

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.5886. Min Act: -0.0000

Data Index: 1631783 (The Pile)

Max Activating Token Index: 660

Text #11

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.5798. Min Act: -0.0000

Data Index: 1450748 (The Pile)

Max Activating Token Index: 110

Text #12

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.4196. Min Act: -0.0000

Data Index: 986308 (The Pile)

Max Activating Token Index: 476

Text #13

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.5109. Min Act: -0.0000

Data Index: 1370651 (The Pile)

Max Activating Token Index: 244

Text #14

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.5838. Min Act: -0.0000

Data Index: 379449 (The Pile)

Max Activating Token Index: 453

Text #15

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.4916. Min Act: -0.0000

Data Index: 993812 (The Pile)

Max Activating Token Index: 454

Text #16

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.4488. Min Act: -0.0000

Data Index: 775840 (The Pile)

Max Activating Token Index: 577

Text #17

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.3505. Min Act: -0.0000

Data Index: 96596 (The Pile)

Max Activating Token Index: 547

Text #18

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.4291. Min Act: -0.0000

Data Index: 600562 (The Pile)

Max Activating Token Index: 841

Text #19

Max Range: 1.8553. Min Range: -1.8553

Max Act: 1.4004. Min Act: -0.0000

Data Index: 1405995 (The Pile)

Max Activating Token Index: 230