Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 5809 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 2.6594. Min Range: -2.6594

Max Act: 2.6594. Min Act: -0.0000

Data Index: 1551256 (Python Code)

Max Activating Token Index: 767

Text #1

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.9994. Min Act: -0.0000

Data Index: 1445794 (Python Code)

Max Activating Token Index: 753

Text #2

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.9377. Min Act: -0.0000

Data Index: 1380346 (Python Code)

Max Activating Token Index: 955

Text #3

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.8599. Min Act: -0.0000

Data Index: 1453915 (Python Code)

Max Activating Token Index: 206

Text #4

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.9542. Min Act: -0.0000

Data Index: 1554640 (Python Code)

Max Activating Token Index: 452

Text #5

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.7584. Min Act: -0.0000

Data Index: 1426857 (Python Code)

Max Activating Token Index: 770

Text #6

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.6598. Min Act: -0.0000

Data Index: 1442869 (Python Code)

Max Activating Token Index: 394

Text #7

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.6494. Min Act: -0.0000

Data Index: 1558368 (Python Code)

Max Activating Token Index: 830

Text #8

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.6376. Min Act: -0.0000

Data Index: 146430 (C4 (Web Text))

Max Activating Token Index: 477

Text #9

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.6135. Min Act: -0.0000

Data Index: 1425196 (Python Code)

Max Activating Token Index: 994

Text #10

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.8775. Min Act: -0.0000

Data Index: 1570612 (Python Code)

Max Activating Token Index: 305

Text #11

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.6794. Min Act: -0.0000

Data Index: 1528526 (Python Code)

Max Activating Token Index: 162

Text #12

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.6830. Min Act: -0.0000

Data Index: 1600083 (Python Code)

Max Activating Token Index: 286

Text #13

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.4838. Min Act: -0.0000

Data Index: 1492863 (Python Code)

Max Activating Token Index: 878

Text #14

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.5835. Min Act: -0.0000

Data Index: 1406508 (Python Code)

Max Activating Token Index: 1009

Text #15

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.4566. Min Act: -0.0000

Data Index: 1453307 (Python Code)

Max Activating Token Index: 418

Text #16

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.6114. Min Act: -0.0000

Data Index: 1497208 (Python Code)

Max Activating Token Index: 1018

Text #17

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.5109. Min Act: -0.0000

Data Index: 1562375 (Python Code)

Max Activating Token Index: 715

Text #18

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.5129. Min Act: -0.0000

Data Index: 1587093 (Python Code)

Max Activating Token Index: 816

Text #19

Max Range: 2.6594. Min Range: -2.6594

Max Act: 1.4903. Min Act: -0.0000

Data Index: 1491354 (Python Code)

Max Activating Token Index: 231