Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2528 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 7.4162. Min Range: -7.4162

Max Act: 7.4162. Min Act: -0.0000

Data Index: 1715910 (The Pile)

Max Activating Token Index: 570

Text #1

Max Range: 7.4162. Min Range: -7.4162

Max Act: 7.3796. Min Act: -0.0000

Data Index: 1811004 (The Pile)

Max Activating Token Index: 423

Text #2

Max Range: 7.4162. Min Range: -7.4162

Max Act: 7.1986. Min Act: -0.0000

Data Index: 634168 (The Pile)

Max Activating Token Index: 475

Text #3

Max Range: 7.4162. Min Range: -7.4162

Max Act: 7.2318. Min Act: -0.0000

Data Index: 1438432 (The Pile)

Max Activating Token Index: 699

Text #4

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.7999. Min Act: -0.0000

Data Index: 1710292 (The Pile)

Max Activating Token Index: 579

Text #5

Max Range: 7.4162. Min Range: -7.4162

Max Act: 7.0056. Min Act: -0.0000

Data Index: 1889850 (The Pile)

Max Activating Token Index: 446

Text #6

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.9849. Min Act: -0.0000

Data Index: 1535851 (The Pile)

Max Activating Token Index: 533

Text #7

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.5015. Min Act: -0.0000

Data Index: 1115220 (The Pile)

Max Activating Token Index: 792

Text #8

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.7294. Min Act: -0.0000

Data Index: 726289 (The Pile)

Max Activating Token Index: 747

Text #9

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.8530. Min Act: -0.0000

Data Index: 1182245 (The Pile)

Max Activating Token Index: 659

Text #10

Max Range: 7.4162. Min Range: -7.4162

Max Act: 7.2345. Min Act: -0.0000

Data Index: 191814 (The Pile)

Max Activating Token Index: 739

Text #11

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.8348. Min Act: -0.0000

Data Index: 757226 (The Pile)

Max Activating Token Index: 628

Text #12

Max Range: 7.4162. Min Range: -7.4162

Max Act: 7.2086. Min Act: -0.0000

Data Index: 72520 (The Pile)

Max Activating Token Index: 548

Text #13

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.4178. Min Act: -0.0000

Data Index: 235034 (The Pile)

Max Activating Token Index: 770

Text #14

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.8147. Min Act: -0.0000

Data Index: 781487 (The Pile)

Max Activating Token Index: 521

Text #15

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.8127. Min Act: -0.0000

Data Index: 594928 (The Pile)

Max Activating Token Index: 433

Text #16

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.7824. Min Act: -0.0000

Data Index: 1329389 (The Pile)

Max Activating Token Index: 500

Text #17

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.3005. Min Act: -0.0000

Data Index: 1385689 (The Pile)

Max Activating Token Index: 735

Text #18

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.6960. Min Act: -0.0000

Data Index: 98184 (The Pile)

Max Activating Token Index: 630

Text #19

Max Range: 7.4162. Min Range: -7.4162

Max Act: 6.2575. Min Act: -0.0000

Data Index: 1394280 (The Pile)

Max Activating Token Index: 794