Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3763 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.6680. Min Act: -0.0000

Data Index: 342733 (The Pile)

Max Activating Token Index: 789

Text #1

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.4436. Min Act: -0.0000

Data Index: 134266 (The Pile)

Max Activating Token Index: 863

Text #2

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.4110. Min Act: -0.0000

Data Index: 1836641 (The Pile)

Max Activating Token Index: 84

Text #3

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.3099. Min Act: -0.0000

Data Index: 911028 (The Pile)

Max Activating Token Index: 96

Text #4

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.2223. Min Act: -0.0000

Data Index: 577176 (The Pile)

Max Activating Token Index: 591

Text #5

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.3404. Min Act: -0.0000

Data Index: 1983880 (The Pile)

Max Activating Token Index: 493

Text #6

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.2611. Min Act: -0.0000

Data Index: 1297209 (The Pile)

Max Activating Token Index: 471

Text #7

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.2563. Min Act: -0.0000

Data Index: 775149 (The Pile)

Max Activating Token Index: 921

Text #8

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.1317. Min Act: -0.0000

Data Index: 1684516 (The Pile)

Max Activating Token Index: 279

Text #9

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.0510. Min Act: -0.0000

Data Index: 1019659 (The Pile)

Max Activating Token Index: 659

Text #10

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.1168. Min Act: -0.0000

Data Index: 1781470 (The Pile)

Max Activating Token Index: 263

Text #11

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.2131. Min Act: 0.0000

Data Index: 1779816 (The Pile)

Max Activating Token Index: 612

Text #12

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.2723. Min Act: -0.0000

Data Index: 1282631 (The Pile)

Max Activating Token Index: 667

Text #13

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.0582. Min Act: -0.0000

Data Index: 583571 (The Pile)

Max Activating Token Index: 543

Text #14

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.1596. Min Act: -0.0000

Data Index: 1406983 (The Pile)

Max Activating Token Index: 984

Text #15

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.1233. Min Act: -0.0000

Data Index: 774783 (The Pile)

Max Activating Token Index: 342

Text #16

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.1372. Min Act: -0.0000

Data Index: 1908844 (The Pile)

Max Activating Token Index: 737

Text #17

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.1923. Min Act: -0.0000

Data Index: 1058484 (The Pile)

Max Activating Token Index: 845

Text #18

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.1084. Min Act: -0.0000

Data Index: 302216 (The Pile)

Max Activating Token Index: 587

Text #19

Max Range: 1.6680. Min Range: -1.6680

Max Act: 1.0661. Min Act: -0.0000

Data Index: 1202128 (The Pile)

Max Activating Token Index: 913