Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 76 in Layer 10

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.7723. Min Act: -0.0000

Data Index: 1595517 (Python Code)

Max Activating Token Index: 229

Text #1

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.4528. Min Act: -0.0000

Data Index: 1394061 (Python Code)

Max Activating Token Index: 975

Text #2

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.4589. Min Act: -0.0000

Data Index: 1428640 (Python Code)

Max Activating Token Index: 603

Text #3

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.3825. Min Act: -0.0000

Data Index: 1554514 (Python Code)

Max Activating Token Index: 140

Text #4

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.7047. Min Act: -0.0000

Data Index: 1408619 (Python Code)

Max Activating Token Index: 248

Text #5

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.1575. Min Act: -0.0000

Data Index: 1395722 (Python Code)

Max Activating Token Index: 319

Text #6

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2544. Min Act: -0.0000

Data Index: 1488849 (Python Code)

Max Activating Token Index: 793

Text #7

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2710. Min Act: -0.0000

Data Index: 1553929 (Python Code)

Max Activating Token Index: 489

Text #8

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.1320. Min Act: -0.0000

Data Index: 1395462 (Python Code)

Max Activating Token Index: 253

Text #9

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2290. Min Act: -0.0000

Data Index: 1637289 (Python Code)

Max Activating Token Index: 381

Text #10

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2595. Min Act: -0.0000

Data Index: 1618591 (Python Code)

Max Activating Token Index: 832

Text #11

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.1813. Min Act: -0.0000

Data Index: 1609779 (Python Code)

Max Activating Token Index: 20

Text #12

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2576. Min Act: -0.0000

Data Index: 1380122 (Python Code)

Max Activating Token Index: 883

Text #13

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.3124. Min Act: -0.0000

Data Index: 1574283 (Python Code)

Max Activating Token Index: 171

Text #14

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.1327. Min Act: -0.0000

Data Index: 1608191 (Python Code)

Max Activating Token Index: 449

Text #15

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2587. Min Act: -0.0000

Data Index: 1611028 (Python Code)

Max Activating Token Index: 57

Text #16

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2984. Min Act: -0.0000

Data Index: 331172 (C4 (Web Text))

Max Activating Token Index: 151

Text #17

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.1893. Min Act: -0.0000

Data Index: 1564126 (Python Code)

Max Activating Token Index: 901

Text #18

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.2619. Min Act: -0.0000

Data Index: 1382827 (Python Code)

Max Activating Token Index: 44

Text #19

Max Range: 1.7723. Min Range: -1.7723

Max Act: 1.0791. Min Act: -0.0000

Data Index: 1386901 (Python Code)

Max Activating Token Index: 902