Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2195 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 7.1329. Min Range: -7.1329

Max Act: 7.0582. Min Act: -0.0000

Data Index: 1795397 (The Pile)

Max Activating Token Index: 404

Text #1

Max Range: 7.1329. Min Range: -7.1329

Max Act: 7.1329. Min Act: -0.0000

Data Index: 810264 (The Pile)

Max Activating Token Index: 164

Text #2

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.7088. Min Act: -0.0000

Data Index: 767446 (The Pile)

Max Activating Token Index: 510

Text #3

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.7060. Min Act: -0.0000

Data Index: 689577 (The Pile)

Max Activating Token Index: 596

Text #4

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.6286. Min Act: -0.0000

Data Index: 1704229 (The Pile)

Max Activating Token Index: 441

Text #5

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.5687. Min Act: -0.0000

Data Index: 610579 (The Pile)

Max Activating Token Index: 172

Text #6

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.2781. Min Act: -0.0000

Data Index: 1651290 (The Pile)

Max Activating Token Index: 780

Text #7

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.5864. Min Act: -0.0000

Data Index: 722431 (The Pile)

Max Activating Token Index: 350

Text #8

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.7659. Min Act: -0.0000

Data Index: 1382143 (The Pile)

Max Activating Token Index: 852

Text #9

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.3733. Min Act: -0.0000

Data Index: 739789 (The Pile)

Max Activating Token Index: 332

Text #10

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.1149. Min Act: -0.0000

Data Index: 535176 (The Pile)

Max Activating Token Index: 172

Text #11

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.8965. Min Act: -0.0000

Data Index: 1290780 (The Pile)

Max Activating Token Index: 586

Text #12

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.4724. Min Act: -0.0000

Data Index: 827463 (The Pile)

Max Activating Token Index: 624

Text #13

Max Range: 7.1329. Min Range: -7.1329

Max Act: 5.9298. Min Act: -0.0000

Data Index: 241822 (The Pile)

Max Activating Token Index: 590

Text #14

Max Range: 7.1329. Min Range: -7.1329

Max Act: 5.8461. Min Act: -0.0000

Data Index: 956728 (The Pile)

Max Activating Token Index: 241

Text #15

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.2103. Min Act: -0.0000

Data Index: 667929 (The Pile)

Max Activating Token Index: 651

Text #16

Max Range: 7.1329. Min Range: -7.1329

Max Act: 6.5657. Min Act: -0.0000

Data Index: 48833 (The Pile)

Max Activating Token Index: 744

Text #17

Max Range: 7.1329. Min Range: -7.1329

Max Act: 5.8235. Min Act: -0.0000

Data Index: 504878 (The Pile)

Max Activating Token Index: 722

Text #18

Max Range: 7.1329. Min Range: -7.1329

Max Act: 5.7694. Min Act: -0.0000

Data Index: 1183980 (The Pile)

Max Activating Token Index: 845

Text #19

Max Range: 7.1329. Min Range: -7.1329

Max Act: 5.9428. Min Act: -0.0000

Data Index: 879338 (The Pile)

Max Activating Token Index: 561