Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2330 in Layer 10

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.8424. Min Act: -0.0000

Data Index: 1579293 (Python Code)

Max Activating Token Index: 266

Text #1

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.6689. Min Act: -0.0000

Data Index: 1520581 (Python Code)

Max Activating Token Index: 493

Text #2

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.4478. Min Act: -0.0000

Data Index: 1548141 (Python Code)

Max Activating Token Index: 421

Text #3

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.2884. Min Act: -0.0000

Data Index: 1414452 (Python Code)

Max Activating Token Index: 198

Text #4

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.2795. Min Act: -0.0000

Data Index: 1369443 (Python Code)

Max Activating Token Index: 347

Text #5

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.3606. Min Act: -0.0000

Data Index: 1649663 (Python Code)

Max Activating Token Index: 286

Text #6

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.3366. Min Act: -0.0000

Data Index: 1539506 (Python Code)

Max Activating Token Index: 372

Text #7

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.4678. Min Act: -0.0000

Data Index: 1565296 (Python Code)

Max Activating Token Index: 519

Text #8

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.1839. Min Act: -0.0000

Data Index: 1366521 (Python Code)

Max Activating Token Index: 438

Text #9

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.2970. Min Act: -0.0000

Data Index: 1476812 (Python Code)

Max Activating Token Index: 370

Text #10

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.2999. Min Act: -0.0000

Data Index: 1492345 (Python Code)

Max Activating Token Index: 646

Text #11

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.2786. Min Act: -0.0000

Data Index: 1410174 (Python Code)

Max Activating Token Index: 467

Text #12

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.4657. Min Act: -0.0000

Data Index: 1425081 (Python Code)

Max Activating Token Index: 587

Text #13

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.2317. Min Act: -0.0000

Data Index: 1562176 (Python Code)

Max Activating Token Index: 277

Text #14

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.1359. Min Act: -0.0000

Data Index: 1432360 (Python Code)

Max Activating Token Index: 716

Text #15

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.0753. Min Act: -0.0000

Data Index: 1511893 (Python Code)

Max Activating Token Index: 405

Text #16

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.3685. Min Act: -0.0000

Data Index: 1552907 (Python Code)

Max Activating Token Index: 487

Text #17

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.2055. Min Act: -0.0000

Data Index: 1479876 (Python Code)

Max Activating Token Index: 797

Text #18

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.1948. Min Act: -0.0000

Data Index: 1583028 (Python Code)

Max Activating Token Index: 380

Text #19

Max Range: 3.8424. Min Range: -3.8424

Max Act: 3.1921. Min Act: -0.0000

Data Index: 1559898 (Python Code)

Max Activating Token Index: 777