Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1350 in Layer 11

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.4429. Min Act: -0.0000

Data Index: 1643141 (Python Code)

Max Activating Token Index: 253

Text #1

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.5207. Min Act: -0.0000

Data Index: 537294 (C4 (Web Text))

Max Activating Token Index: 442

Text #2

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.2541. Min Act: -0.0000

Data Index: 1623606 (Python Code)

Max Activating Token Index: 693

Text #3

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.3193. Min Act: -0.0000

Data Index: 1362521 (Python Code)

Max Activating Token Index: 269

Text #4

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.0918. Min Act: -0.0000

Data Index: 1610048 (Python Code)

Max Activating Token Index: 854

Text #5

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.2192. Min Act: -0.0000

Data Index: 1428487 (Python Code)

Max Activating Token Index: 645

Text #6

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1262. Min Act: -0.0000

Data Index: 295729 (C4 (Web Text))

Max Activating Token Index: 510

Text #7

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1856. Min Act: -0.0000

Data Index: 1620040 (Python Code)

Max Activating Token Index: 906

Text #8

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.2264. Min Act: -0.0000

Data Index: 1409768 (Python Code)

Max Activating Token Index: 839

Text #9

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1917. Min Act: -0.0000

Data Index: 1454432 (Python Code)

Max Activating Token Index: 618

Text #10

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.0455. Min Act: -0.0000

Data Index: 1258842 (C4 (Web Text))

Max Activating Token Index: 498

Text #11

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1694. Min Act: -0.0000

Data Index: 1561974 (Python Code)

Max Activating Token Index: 947

Text #12

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1292. Min Act: -0.0000

Data Index: 1433027 (Python Code)

Max Activating Token Index: 851

Text #13

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1692. Min Act: -0.0000

Data Index: 1489080 (Python Code)

Max Activating Token Index: 600

Text #14

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1609. Min Act: -0.0000

Data Index: 1607033 (Python Code)

Max Activating Token Index: 953

Text #15

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1572. Min Act: -0.0000

Data Index: 1557614 (Python Code)

Max Activating Token Index: 454

Text #16

Max Range: 1.5207. Min Range: -1.5207

Max Act: 0.9930. Min Act: -0.0000

Data Index: 1383299 (Python Code)

Max Activating Token Index: 955

Text #17

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1182. Min Act: -0.0000

Data Index: 1510418 (Python Code)

Max Activating Token Index: 808

Text #18

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1505. Min Act: -0.0000

Data Index: 1377210 (Python Code)

Max Activating Token Index: 105

Text #19

Max Range: 1.5207. Min Range: -1.5207

Max Act: 1.1014. Min Act: -0.0000

Data Index: 1424201 (Python Code)

Max Activating Token Index: 94