Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4311 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.4308. Min Act: -0.0001

Data Index: 1462126 (Python Code)

Max Activating Token Index: 73

Text #1

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.5756. Min Act: -0.0000

Data Index: 1450673 (Python Code)

Max Activating Token Index: 801

Text #2

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.4660. Min Act: -0.0001

Data Index: 1552670 (Python Code)

Max Activating Token Index: 991

Text #3

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.4300. Min Act: -0.0001

Data Index: 1487738 (Python Code)

Max Activating Token Index: 131

Text #4

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.3633. Min Act: -0.0001

Data Index: 1634883 (Python Code)

Max Activating Token Index: 470

Text #5

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.2994. Min Act: -0.0001

Data Index: 1388240 (Python Code)

Max Activating Token Index: 738

Text #6

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.2570. Min Act: -0.0001

Data Index: 1576400 (Python Code)

Max Activating Token Index: 120

Text #7

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.2424. Min Act: -0.0001

Data Index: 1406451 (Python Code)

Max Activating Token Index: 510

Text #8

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1662. Min Act: -0.0001

Data Index: 1648046 (Python Code)

Max Activating Token Index: 842

Text #9

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.2242. Min Act: -0.0001

Data Index: 1548605 (Python Code)

Max Activating Token Index: 260

Text #10

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.2710. Min Act: -0.0001

Data Index: 1516321 (Python Code)

Max Activating Token Index: 611

Text #11

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1802. Min Act: -0.0001

Data Index: 1364984 (Python Code)

Max Activating Token Index: 703

Text #12

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1034. Min Act: -0.0000

Data Index: 1537619 (Python Code)

Max Activating Token Index: 228

Text #13

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.2629. Min Act: -0.0000

Data Index: 1519149 (Python Code)

Max Activating Token Index: 885

Text #14

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1425. Min Act: -0.0001

Data Index: 1372238 (Python Code)

Max Activating Token Index: 541

Text #15

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1703. Min Act: -0.0001

Data Index: 1542034 (Python Code)

Max Activating Token Index: 624

Text #16

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1367. Min Act: -0.0001

Data Index: 1394841 (Python Code)

Max Activating Token Index: 317

Text #17

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1419. Min Act: -0.0001

Data Index: 1369509 (Python Code)

Max Activating Token Index: 71

Text #18

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1684. Min Act: -0.0001

Data Index: 1363055 (Python Code)

Max Activating Token Index: 881

Text #19

Max Range: 1.5756. Min Range: -1.5756

Max Act: 1.1300. Min Act: -0.0001

Data Index: 1491046 (Python Code)

Max Activating Token Index: 198