Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 652 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.1597. Min Act: -0.0000

Data Index: 1510965 (Python Code)

Max Activating Token Index: 443

Text #1

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.1999. Min Act: -0.0000

Data Index: 1441763 (Python Code)

Max Activating Token Index: 582

Text #2

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.1612. Min Act: -0.0000

Data Index: 1402702 (Python Code)

Max Activating Token Index: 51

Text #3

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.1129. Min Act: -0.0000

Data Index: 1603000 (Python Code)

Max Activating Token Index: 305

Text #4

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.0784. Min Act: -0.0000

Data Index: 1447090 (Python Code)

Max Activating Token Index: 803

Text #5

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.0281. Min Act: -0.0000

Data Index: 1385831 (Python Code)

Max Activating Token Index: 809

Text #6

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9737. Min Act: -0.0000

Data Index: 1530298 (Python Code)

Max Activating Token Index: 500

Text #7

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9652. Min Act: -0.0000

Data Index: 1379671 (Python Code)

Max Activating Token Index: 185

Text #8

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.8356. Min Act: -0.0000

Data Index: 1526541 (Python Code)

Max Activating Token Index: 975

Text #9

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9045. Min Act: -0.0000

Data Index: 1432724 (Python Code)

Max Activating Token Index: 481

Text #10

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.0128. Min Act: -0.0000

Data Index: 1567545 (Python Code)

Max Activating Token Index: 41

Text #11

Max Range: 1.1999. Min Range: -1.1999

Max Act: 1.0095. Min Act: -0.0000

Data Index: 1530566 (Python Code)

Max Activating Token Index: 282

Text #12

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9108. Min Act: -0.0000

Data Index: 1414279 (Python Code)

Max Activating Token Index: 56

Text #13

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9363. Min Act: -0.0000

Data Index: 1417415 (Python Code)

Max Activating Token Index: 195

Text #14

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9403. Min Act: -0.0000

Data Index: 1372515 (Python Code)

Max Activating Token Index: 158

Text #15

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9944. Min Act: -0.0000

Data Index: 1634310 (Python Code)

Max Activating Token Index: 710

Text #16

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9000. Min Act: -0.0000

Data Index: 1548750 (Python Code)

Max Activating Token Index: 370

Text #17

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.8215. Min Act: -0.0000

Data Index: 1516630 (Python Code)

Max Activating Token Index: 289

Text #18

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.9006. Min Act: -0.0000

Data Index: 1378024 (Python Code)

Max Activating Token Index: 211

Text #19

Max Range: 1.1999. Min Range: -1.1999

Max Act: 0.8817. Min Act: -0.0000

Data Index: 1508295 (Python Code)

Max Activating Token Index: 143