Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2159 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.3151. Min Act: -0.0000

Data Index: 477161 (The Pile)

Max Activating Token Index: 695

Text #1

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2758. Min Act: -0.0000

Data Index: 778178 (The Pile)

Max Activating Token Index: 872

Text #2

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2877. Min Act: -0.0000

Data Index: 1363589 (The Pile)

Max Activating Token Index: 498

Text #3

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2751. Min Act: -0.0000

Data Index: 649621 (The Pile)

Max Activating Token Index: 533

Text #4

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2701. Min Act: -0.0000

Data Index: 904465 (The Pile)

Max Activating Token Index: 449

Text #5

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2643. Min Act: -0.0000

Data Index: 1137623 (The Pile)

Max Activating Token Index: 411

Text #6

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2709. Min Act: -0.0000

Data Index: 1215174 (The Pile)

Max Activating Token Index: 109

Text #7

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2486. Min Act: -0.0000

Data Index: 82909 (The Pile)

Max Activating Token Index: 855

Text #8

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2549. Min Act: -0.0000

Data Index: 453548 (The Pile)

Max Activating Token Index: 405

Text #9

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2407. Min Act: -0.0000

Data Index: 1565606 (The Pile)

Max Activating Token Index: 555

Text #10

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2230. Min Act: -0.0000

Data Index: 1386977 (The Pile)

Max Activating Token Index: 716

Text #11

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2498. Min Act: -0.0000

Data Index: 596860 (The Pile)

Max Activating Token Index: 492

Text #12

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2453. Min Act: -0.0000

Data Index: 223533 (The Pile)

Max Activating Token Index: 627

Text #13

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2358. Min Act: -0.0000

Data Index: 1836161 (The Pile)

Max Activating Token Index: 411

Text #14

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2535. Min Act: -0.0000

Data Index: 1594501 (The Pile)

Max Activating Token Index: 473

Text #15

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2445. Min Act: -0.0000

Data Index: 1955725 (The Pile)

Max Activating Token Index: 431

Text #16

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2644. Min Act: -0.0000

Data Index: 782854 (The Pile)

Max Activating Token Index: 160

Text #17

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2247. Min Act: -0.0000

Data Index: 663450 (The Pile)

Max Activating Token Index: 814

Text #18

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2373. Min Act: -0.0000

Data Index: 951484 (The Pile)

Max Activating Token Index: 433

Text #19

Max Range: 0.3151. Min Range: -0.3151

Max Act: 0.2329. Min Act: -0.0000

Data Index: 1665652 (The Pile)

Max Activating Token Index: 499