Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 3169 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.2732. Min Range: -1.2732

Max Act: 1.2732. Min Act: -0.0000

Data Index: 172630 (C4 (Web Text))

Max Activating Token Index: 915

Text #1

Max Range: 1.2732. Min Range: -1.2732

Max Act: 1.1093. Min Act: -0.0000

Data Index: 534991 (C4 (Web Text))

Max Activating Token Index: 409

Text #2

Max Range: 1.2732. Min Range: -1.2732

Max Act: 1.1332. Min Act: -0.0000

Data Index: 353529 (C4 (Web Text))

Max Activating Token Index: 513

Text #3

Max Range: 1.2732. Min Range: -1.2732

Max Act: 1.0851. Min Act: -0.0000

Data Index: 541275 (C4 (Web Text))

Max Activating Token Index: 570

Text #4

Max Range: 1.2732. Min Range: -1.2732

Max Act: 1.0514. Min Act: -0.0000

Data Index: 1081265 (C4 (Web Text))

Max Activating Token Index: 202

Text #5

Max Range: 1.2732. Min Range: -1.2732

Max Act: 1.0334. Min Act: -0.0000

Data Index: 806200 (C4 (Web Text))

Max Activating Token Index: 137

Text #6

Max Range: 1.2732. Min Range: -1.2732

Max Act: 1.0323. Min Act: -0.0000

Data Index: 457419 (C4 (Web Text))

Max Activating Token Index: 791

Text #7

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.9368. Min Act: -0.0000

Data Index: 1252239 (C4 (Web Text))

Max Activating Token Index: 995

Text #8

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.9853. Min Act: -0.0000

Data Index: 1475682 (Python Code)

Max Activating Token Index: 583

Text #9

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.8494. Min Act: -0.0000

Data Index: 1614712 (Python Code)

Max Activating Token Index: 769

Text #10

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.9074. Min Act: -0.0000

Data Index: 1273035 (C4 (Web Text))

Max Activating Token Index: 1001

Text #11

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.9542. Min Act: -0.0000

Data Index: 1532595 (Python Code)

Max Activating Token Index: 140

Text #12

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.8296. Min Act: -0.0000

Data Index: 335719 (C4 (Web Text))

Max Activating Token Index: 468

Text #13

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.8128. Min Act: -0.0000

Data Index: 684511 (C4 (Web Text))

Max Activating Token Index: 578

Text #14

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.7770. Min Act: -0.0000

Data Index: 1495842 (Python Code)

Max Activating Token Index: 719

Text #15

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.7841. Min Act: -0.0000

Data Index: 1431289 (Python Code)

Max Activating Token Index: 817

Text #16

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.7914. Min Act: -0.0000

Data Index: 1624424 (Python Code)

Max Activating Token Index: 678

Text #17

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.7243. Min Act: -0.0000

Data Index: 1481857 (Python Code)

Max Activating Token Index: 696

Text #18

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.8053. Min Act: -0.0000

Data Index: 988917 (C4 (Web Text))

Max Activating Token Index: 69

Text #19

Max Range: 1.2732. Min Range: -1.2732

Max Act: 0.7540. Min Act: -0.0000

Data Index: 975116 (C4 (Web Text))

Max Activating Token Index: 628