Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1361 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 2.1217. Min Range: -2.1217

Max Act: 2.1217. Min Act: -0.0000

Data Index: 1604981 (Python Code)

Max Activating Token Index: 566

Text #1

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.9335. Min Act: -0.0000

Data Index: 1620411 (Python Code)

Max Activating Token Index: 84

Text #2

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.7128. Min Act: -0.0000

Data Index: 1580897 (Python Code)

Max Activating Token Index: 972

Text #3

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.8315. Min Act: -0.0000

Data Index: 1642952 (Python Code)

Max Activating Token Index: 685

Text #4

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.6973. Min Act: -0.0000

Data Index: 1380743 (Python Code)

Max Activating Token Index: 494

Text #5

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.6824. Min Act: -0.0000

Data Index: 1381143 (Python Code)

Max Activating Token Index: 706

Text #6

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.5573. Min Act: -0.0000

Data Index: 1379516 (Python Code)

Max Activating Token Index: 107

Text #7

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.6171. Min Act: -0.0000

Data Index: 1619477 (Python Code)

Max Activating Token Index: 135

Text #8

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.6024. Min Act: -0.0000

Data Index: 1481680 (Python Code)

Max Activating Token Index: 190

Text #9

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.5807. Min Act: -0.0000

Data Index: 1423431 (Python Code)

Max Activating Token Index: 98

Text #10

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.6635. Min Act: -0.0000

Data Index: 1457673 (Python Code)

Max Activating Token Index: 199

Text #11

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.5655. Min Act: -0.0000

Data Index: 1359898 (Python Code)

Max Activating Token Index: 103

Text #12

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.6522. Min Act: -0.0000

Data Index: 1636140 (Python Code)

Max Activating Token Index: 752

Text #13

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.5399. Min Act: -0.0000

Data Index: 1633163 (Python Code)

Max Activating Token Index: 129

Text #14

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.4818. Min Act: -0.0000

Data Index: 1360458 (Python Code)

Max Activating Token Index: 866

Text #15

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.5544. Min Act: -0.0000

Data Index: 1540067 (Python Code)

Max Activating Token Index: 369

Text #16

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.4781. Min Act: -0.0000

Data Index: 1635892 (Python Code)

Max Activating Token Index: 79

Text #17

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.4602. Min Act: -0.0000

Data Index: 1507322 (Python Code)

Max Activating Token Index: 293

Text #18

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.5396. Min Act: -0.0000

Data Index: 1532032 (Python Code)

Max Activating Token Index: 282

Text #19

Max Range: 2.1217. Min Range: -2.1217

Max Act: 1.4499. Min Act: -0.0000

Data Index: 1460866 (Python Code)

Max Activating Token Index: 97