Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2119 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1861. Min Act: -0.0000

Data Index: 214054 (C4 (Web Text))

Max Activating Token Index: 71

Text #1

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1850. Min Act: -0.0000

Data Index: 40318 (C4 (Web Text))

Max Activating Token Index: 515

Text #2

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1901. Min Act: -0.0000

Data Index: 695023 (C4 (Web Text))

Max Activating Token Index: 852

Text #3

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1760. Min Act: -0.0000

Data Index: 355829 (C4 (Web Text))

Max Activating Token Index: 357

Text #4

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1799. Min Act: -0.0000

Data Index: 998108 (C4 (Web Text))

Max Activating Token Index: 161

Text #5

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1767. Min Act: -0.0000

Data Index: 608929 (C4 (Web Text))

Max Activating Token Index: 689

Text #6

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1626. Min Act: -0.0000

Data Index: 332449 (C4 (Web Text))

Max Activating Token Index: 713

Text #7

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1605. Min Act: -0.0000

Data Index: 1147478 (C4 (Web Text))

Max Activating Token Index: 761

Text #8

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1656. Min Act: 0.0000

Data Index: 1266732 (C4 (Web Text))

Max Activating Token Index: 59

Text #9

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1654. Min Act: -0.0000

Data Index: 534326 (C4 (Web Text))

Max Activating Token Index: 36

Text #10

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1650. Min Act: -0.0000

Data Index: 339465 (C4 (Web Text))

Max Activating Token Index: 566

Text #11

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1624. Min Act: 0.0000

Data Index: 770231 (C4 (Web Text))

Max Activating Token Index: 886

Text #12

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1553. Min Act: -0.0000

Data Index: 1259646 (C4 (Web Text))

Max Activating Token Index: 850

Text #13

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1598. Min Act: -0.0000

Data Index: 880292 (C4 (Web Text))

Max Activating Token Index: 543

Text #14

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1589. Min Act: -0.0000

Data Index: 1284525 (C4 (Web Text))

Max Activating Token Index: 505

Text #15

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1585. Min Act: -0.0000

Data Index: 1146556 (C4 (Web Text))

Max Activating Token Index: 645

Text #16

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1562. Min Act: -0.0000

Data Index: 1236165 (C4 (Web Text))

Max Activating Token Index: 279

Text #17

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1624. Min Act: -0.0000

Data Index: 1223481 (C4 (Web Text))

Max Activating Token Index: 433

Text #18

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1586. Min Act: -0.0000

Data Index: 98863 (C4 (Web Text))

Max Activating Token Index: 402

Text #19

Max Range: 0.1901. Min Range: -0.1901

Max Act: 0.1593. Min Act: -0.0000

Data Index: 59873 (C4 (Web Text))

Max Activating Token Index: 104