Model: SoLU Model: 6 Layers, 3072 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1287 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-6l')

Text #0

Max Range: 1.1561. Min Range: -1.1561

Max Act: 1.0849. Min Act: -0.0004

Data Index: 1587039 (Python Code)

Max Activating Token Index: 3

Text #1

Max Range: 1.1561. Min Range: -1.1561

Max Act: 1.0198. Min Act: -0.0004

Data Index: 1483272 (Python Code)

Max Activating Token Index: 933

Text #2

Max Range: 1.1561. Min Range: -1.1561

Max Act: 1.0512. Min Act: -0.0004

Data Index: 1527698 (Python Code)

Max Activating Token Index: 7

Text #3

Max Range: 1.1561. Min Range: -1.1561

Max Act: 1.0015. Min Act: -0.0004

Data Index: 1456378 (Python Code)

Max Activating Token Index: 8

Text #4

Max Range: 1.1561. Min Range: -1.1561

Max Act: 1.0117. Min Act: -0.0004

Data Index: 1443787 (Python Code)

Max Activating Token Index: 7

Text #5

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9834. Min Act: -0.0004

Data Index: 1502491 (Python Code)

Max Activating Token Index: 5

Text #6

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9681. Min Act: -0.0004

Data Index: 1583106 (Python Code)

Max Activating Token Index: 8

Text #7

Max Range: 1.1561. Min Range: -1.1561

Max Act: 1.1561. Min Act: -0.0004

Data Index: 1506030 (Python Code)

Max Activating Token Index: 290

Text #8

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9901. Min Act: -0.0004

Data Index: 1522501 (Python Code)

Max Activating Token Index: 6

Text #9

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9880. Min Act: -0.0004

Data Index: 1514270 (Python Code)

Max Activating Token Index: 10

Text #10

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.8663. Min Act: -0.0004

Data Index: 1494354 (Python Code)

Max Activating Token Index: 871

Text #11

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.8540. Min Act: -0.0004

Data Index: 1597724 (Python Code)

Max Activating Token Index: 24

Text #12

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.8502. Min Act: -0.0004

Data Index: 1641978 (Python Code)

Max Activating Token Index: 296

Text #13

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9891. Min Act: -0.0004

Data Index: 1649062 (Python Code)

Max Activating Token Index: 416

Text #14

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.8507. Min Act: -0.0004

Data Index: 1431329 (Python Code)

Max Activating Token Index: 574

Text #15

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9795. Min Act: -0.0004

Data Index: 1484196 (Python Code)

Max Activating Token Index: 273

Text #16

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9819. Min Act: -0.0004

Data Index: 1620276 (Python Code)

Max Activating Token Index: 8

Text #17

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.8901. Min Act: -0.0004

Data Index: 1507793 (Python Code)

Max Activating Token Index: 222

Text #18

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.9313. Min Act: -0.0004

Data Index: 1539336 (Python Code)

Max Activating Token Index: 9

Text #19

Max Range: 1.1561. Min Range: -1.1561

Max Act: 0.8464. Min Act: -0.0004

Data Index: 1583821 (Python Code)

Max Activating Token Index: 10