Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4880 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 3.3555. Min Range: -3.3555

Max Act: 3.3555. Min Act: -0.0000

Data Index: 1588247 (Python Code)

Max Activating Token Index: 502

Text #1

Max Range: 3.3555. Min Range: -3.3555

Max Act: 3.2253. Min Act: -0.0000

Data Index: 1459769 (Python Code)

Max Activating Token Index: 438

Text #2

Max Range: 3.3555. Min Range: -3.3555

Max Act: 2.9715. Min Act: -0.0000

Data Index: 1591565 (Python Code)

Max Activating Token Index: 735

Text #3

Max Range: 3.3555. Min Range: -3.3555

Max Act: 2.7775. Min Act: -0.0000

Data Index: 1448445 (Python Code)

Max Activating Token Index: 98

Text #4

Max Range: 3.3555. Min Range: -3.3555

Max Act: 2.7757. Min Act: -0.0000

Data Index: 1477844 (Python Code)

Max Activating Token Index: 969

Text #5

Max Range: 3.3555. Min Range: -3.3555

Max Act: 2.2276. Min Act: -0.0000

Data Index: 1554073 (Python Code)

Max Activating Token Index: 1017

Text #6

Max Range: 3.3555. Min Range: -3.3555

Max Act: 2.3375. Min Act: -0.0000

Data Index: 1415892 (Python Code)

Max Activating Token Index: 893

Text #7

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.9799. Min Act: -0.0000

Data Index: 1535637 (Python Code)

Max Activating Token Index: 396

Text #8

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.9927. Min Act: -0.0000

Data Index: 1404740 (Python Code)

Max Activating Token Index: 634

Text #9

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.9746. Min Act: -0.0000

Data Index: 1500050 (Python Code)

Max Activating Token Index: 913

Text #10

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.8511. Min Act: -0.0000

Data Index: 1426646 (Python Code)

Max Activating Token Index: 781

Text #11

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.7264. Min Act: -0.0000

Data Index: 1394040 (Python Code)

Max Activating Token Index: 64

Text #12

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.7985. Min Act: -0.0000

Data Index: 135559 (C4 (Web Text))

Max Activating Token Index: 716

Text #13

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.7640. Min Act: -0.0000

Data Index: 21155 (C4 (Web Text))

Max Activating Token Index: 935

Text #14

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.7811. Min Act: -0.0000

Data Index: 1402512 (Python Code)

Max Activating Token Index: 669

Text #15

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.8593. Min Act: -0.0000

Data Index: 1602884 (Python Code)

Max Activating Token Index: 669

Text #16

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.8371. Min Act: -0.0000

Data Index: 1655385 (Python Code)

Max Activating Token Index: 865

Text #17

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.7157. Min Act: -0.0000

Data Index: 1290351 (C4 (Web Text))

Max Activating Token Index: 654

Text #18

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.5289. Min Act: -0.0000

Data Index: 1390690 (Python Code)

Max Activating Token Index: 656

Text #19

Max Range: 3.3555. Min Range: -3.3555

Max Act: 1.5669. Min Act: -0.0000

Data Index: 1608826 (Python Code)

Max Activating Token Index: 268