Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4348 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!

Text #0

Max Range: 1.1555. Min Range: -1.1555

Max Act: 1.1555. Min Act: -0.0002

Data Index: 440932 (C4 (Web Text))

Max Activating Token Index: 200

Text #1

Max Range: 1.1555. Min Range: -1.1555

Max Act: 1.0719. Min Act: -0.0002

Data Index: 398772 (C4 (Web Text))

Max Activating Token Index: 642

Text #2

Max Range: 1.1555. Min Range: -1.1555

Max Act: 1.0028. Min Act: -0.0002

Data Index: 4334 (C4 (Web Text))

Max Activating Token Index: 470

Text #3

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9767. Min Act: -0.0002

Data Index: 92057 (C4 (Web Text))

Max Activating Token Index: 668

Text #4

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9715. Min Act: -0.0002

Data Index: 135463 (C4 (Web Text))

Max Activating Token Index: 345

Text #5

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9501. Min Act: -0.0002

Data Index: 193814 (C4 (Web Text))

Max Activating Token Index: 758

Text #6

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9514. Min Act: -0.0002

Data Index: 34969 (C4 (Web Text))

Max Activating Token Index: 778

Text #7

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9711. Min Act: -0.0002

Data Index: 1061442 (C4 (Web Text))

Max Activating Token Index: 457

Text #8

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9510. Min Act: -0.0002

Data Index: 154809 (C4 (Web Text))

Max Activating Token Index: 376

Text #9

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9573. Min Act: -0.0002

Data Index: 34573 (C4 (Web Text))

Max Activating Token Index: 528

Text #10

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.9018. Min Act: -0.0002

Data Index: 1169457 (C4 (Web Text))

Max Activating Token Index: 416

Text #11

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.8709. Min Act: -0.0002

Data Index: 1324887 (C4 (Web Text))

Max Activating Token Index: 94

Text #12

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.8481. Min Act: -0.0002

Data Index: 10125 (C4 (Web Text))

Max Activating Token Index: 133

Text #13

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.8603. Min Act: -0.0002

Data Index: 649173 (C4 (Web Text))

Max Activating Token Index: 511

Text #14

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.8543. Min Act: -0.0002

Data Index: 19558 (C4 (Web Text))

Max Activating Token Index: 636

Text #15

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.8248. Min Act: -0.0002

Data Index: 1251637 (C4 (Web Text))

Max Activating Token Index: 18

Text #16

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.8388. Min Act: -0.0002

Data Index: 618433 (C4 (Web Text))

Max Activating Token Index: 827

Text #17

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.8240. Min Act: -0.0002

Data Index: 897357 (C4 (Web Text))

Max Activating Token Index: 28

Text #18

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.7906. Min Act: -0.0002

Data Index: 136215 (C4 (Web Text))

Max Activating Token Index: 185

Text #19

Max Range: 1.1555. Min Range: -1.1555

Max Act: 0.7871. Min Act: -0.0002

Data Index: 502021 (C4 (Web Text))

Max Activating Token Index: 412