Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 5254 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 3.2074. Min Range: -3.2074

Max Act: 3.2074. Min Act: -0.0000

Data Index: 1429152 (Python Code)

Max Activating Token Index: 114

Text #1

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.7730. Min Act: -0.0000

Data Index: 1487355 (Python Code)

Max Activating Token Index: 54

Text #2

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.4140. Min Act: -0.0000

Data Index: 1447061 (Python Code)

Max Activating Token Index: 684

Text #3

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.4290. Min Act: -0.0000

Data Index: 1442340 (Python Code)

Max Activating Token Index: 784

Text #4

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.2596. Min Act: -0.0000

Data Index: 1641169 (Python Code)

Max Activating Token Index: 199

Text #5

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.2341. Min Act: -0.0000

Data Index: 1646549 (Python Code)

Max Activating Token Index: 632

Text #6

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.1195. Min Act: -0.0000

Data Index: 1403848 (Python Code)

Max Activating Token Index: 681

Text #7

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.2455. Min Act: -0.0000

Data Index: 1439648 (Python Code)

Max Activating Token Index: 83

Text #8

Max Range: 3.2074. Min Range: -3.2074

Max Act: 2.0731. Min Act: -0.0000

Data Index: 1476919 (Python Code)

Max Activating Token Index: 345

Text #9

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.8072. Min Act: -0.0000

Data Index: 1619010 (Python Code)

Max Activating Token Index: 619

Text #10

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.7756. Min Act: -0.0000

Data Index: 1581279 (Python Code)

Max Activating Token Index: 976

Text #11

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.9960. Min Act: -0.0000

Data Index: 1570818 (Python Code)

Max Activating Token Index: 425

Text #12

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.7842. Min Act: -0.0000

Data Index: 1449633 (Python Code)

Max Activating Token Index: 645

Text #13

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.8939. Min Act: -0.0000

Data Index: 1617054 (Python Code)

Max Activating Token Index: 494

Text #14

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.8907. Min Act: -0.0000

Data Index: 1485085 (Python Code)

Max Activating Token Index: 832

Text #15

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.7400. Min Act: -0.0000

Data Index: 1414333 (Python Code)

Max Activating Token Index: 246

Text #16

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.7249. Min Act: -0.0000

Data Index: 1488673 (Python Code)

Max Activating Token Index: 944

Text #17

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.6800. Min Act: -0.0000

Data Index: 1527699 (Python Code)

Max Activating Token Index: 316

Text #18

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.6840. Min Act: -0.0000

Data Index: 1445586 (Python Code)

Max Activating Token Index: 81

Text #19

Max Range: 3.2074. Min Range: -3.2074

Max Act: 1.8729. Min Act: -0.0000

Data Index: 1644530 (Python Code)

Max Activating Token Index: 814