Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1343 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 3.6509. Min Range: -3.6509

Max Act: 3.6509. Min Act: -0.0000

Data Index: 1590266 (Python Code)

Max Activating Token Index: 855

Text #1

Max Range: 3.6509. Min Range: -3.6509

Max Act: 3.0546. Min Act: -0.0000

Data Index: 1532122 (Python Code)

Max Activating Token Index: 452

Text #2

Max Range: 3.6509. Min Range: -3.6509

Max Act: 3.0287. Min Act: -0.0000

Data Index: 1611558 (Python Code)

Max Activating Token Index: 765

Text #3

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.8478. Min Act: -0.0000

Data Index: 1400370 (Python Code)

Max Activating Token Index: 460

Text #4

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.7628. Min Act: -0.0000

Data Index: 1589269 (Python Code)

Max Activating Token Index: 527

Text #5

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.9047. Min Act: -0.0000

Data Index: 1459240 (Python Code)

Max Activating Token Index: 433

Text #6

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.8755. Min Act: -0.0000

Data Index: 1376593 (Python Code)

Max Activating Token Index: 832

Text #7

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.7649. Min Act: -0.0000

Data Index: 1635451 (Python Code)

Max Activating Token Index: 239

Text #8

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.5995. Min Act: -0.0000

Data Index: 1439445 (Python Code)

Max Activating Token Index: 573

Text #9

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.6216. Min Act: -0.0000

Data Index: 1435069 (Python Code)

Max Activating Token Index: 778

Text #10

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.7196. Min Act: -0.0000

Data Index: 1553627 (Python Code)

Max Activating Token Index: 611

Text #11

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.7210. Min Act: -0.0000

Data Index: 1443046 (Python Code)

Max Activating Token Index: 880

Text #12

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.8607. Min Act: -0.0000

Data Index: 1523054 (Python Code)

Max Activating Token Index: 459

Text #13

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.5543. Min Act: -0.0000

Data Index: 1598331 (Python Code)

Max Activating Token Index: 45

Text #14

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.6833. Min Act: -0.0000

Data Index: 1649777 (Python Code)

Max Activating Token Index: 525

Text #15

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.6690. Min Act: -0.0000

Data Index: 1644587 (Python Code)

Max Activating Token Index: 437

Text #16

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.5473. Min Act: -0.0000

Data Index: 1422702 (Python Code)

Max Activating Token Index: 685

Text #17

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.7612. Min Act: -0.0000

Data Index: 1524456 (Python Code)

Max Activating Token Index: 390

Text #18

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.7626. Min Act: -0.0000

Data Index: 1411218 (Python Code)

Max Activating Token Index: 307

Text #19

Max Range: 3.6509. Min Range: -3.6509

Max Act: 2.6094. Min Act: -0.0000

Data Index: 1375554 (Python Code)

Max Activating Token Index: 712