Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2459 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7743. Min Act: -0.0000

Data Index: 1400143 (Python Code)

Max Activating Token Index: 653

Text #1

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7327. Min Act: -0.0000

Data Index: 1446403 (Python Code)

Max Activating Token Index: 674

Text #2

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7522. Min Act: -0.0000

Data Index: 1570864 (Python Code)

Max Activating Token Index: 707

Text #3

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7217. Min Act: -0.0000

Data Index: 1401321 (Python Code)

Max Activating Token Index: 991

Text #4

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7126. Min Act: -0.0000

Data Index: 1456105 (Python Code)

Max Activating Token Index: 399

Text #5

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7233. Min Act: -0.0000

Data Index: 1478183 (Python Code)

Max Activating Token Index: 339

Text #6

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7078. Min Act: -0.0000

Data Index: 1367709 (Python Code)

Max Activating Token Index: 1019

Text #7

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7400. Min Act: -0.0000

Data Index: 1455544 (Python Code)

Max Activating Token Index: 265

Text #8

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7051. Min Act: -0.0000

Data Index: 1426978 (Python Code)

Max Activating Token Index: 917

Text #9

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6165. Min Act: -0.0000

Data Index: 1521466 (Python Code)

Max Activating Token Index: 1006

Text #10

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6324. Min Act: -0.0000

Data Index: 1399924 (Python Code)

Max Activating Token Index: 955

Text #11

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6517. Min Act: -0.0000

Data Index: 1646382 (Python Code)

Max Activating Token Index: 536

Text #12

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6027. Min Act: -0.0000

Data Index: 1617718 (Python Code)

Max Activating Token Index: 975

Text #13

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6606. Min Act: -0.0000

Data Index: 1378070 (Python Code)

Max Activating Token Index: 976

Text #14

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6115. Min Act: -0.0000

Data Index: 1466547 (Python Code)

Max Activating Token Index: 307

Text #15

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6569. Min Act: -0.0000

Data Index: 1583885 (Python Code)

Max Activating Token Index: 812

Text #16

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.5958. Min Act: -0.0000

Data Index: 1419951 (Python Code)

Max Activating Token Index: 152

Text #17

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.7004. Min Act: 0.0000

Data Index: 1546110 (Python Code)

Max Activating Token Index: 753

Text #18

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.6639. Min Act: -0.0000

Data Index: 1589585 (Python Code)

Max Activating Token Index: 184

Text #19

Max Range: 0.7743. Min Range: -0.7743

Max Act: 0.5411. Min Act: -0.0000

Data Index: 1422800 (Python Code)

Max Activating Token Index: 98