Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 752 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l')

Text #0

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4268. Min Act: -0.0000

Data Index: 1430436 (Python Code)

Max Activating Token Index: 733

Text #1

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.5484. Min Act: -0.0000

Data Index: 1557669 (Python Code)

Max Activating Token Index: 892

Text #2

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4237. Min Act: -0.0000

Data Index: 1493862 (Python Code)

Max Activating Token Index: 984

Text #3

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.5674. Min Act: -0.0000

Data Index: 1489966 (Python Code)

Max Activating Token Index: 910

Text #4

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.6451. Min Act: -0.0000

Data Index: 1616409 (Python Code)

Max Activating Token Index: 1013

Text #5

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.5138. Min Act: -0.0000

Data Index: 1497381 (Python Code)

Max Activating Token Index: 999

Text #6

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.5982. Min Act: -0.0000

Data Index: 1638429 (Python Code)

Max Activating Token Index: 959

Text #7

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.2866. Min Act: -0.0000

Data Index: 1645087 (Python Code)

Max Activating Token Index: 803

Text #8

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.6625. Min Act: -0.0000

Data Index: 1574688 (Python Code)

Max Activating Token Index: 830

Text #9

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.5752. Min Act: -0.0000

Data Index: 1372040 (Python Code)

Max Activating Token Index: 1019

Text #10

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.6165. Min Act: -0.0000

Data Index: 1514282 (Python Code)

Max Activating Token Index: 5

Text #11

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4332. Min Act: -0.0000

Data Index: 1603391 (Python Code)

Max Activating Token Index: 818

Text #12

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4399. Min Act: -0.0000

Data Index: 1618179 (Python Code)

Max Activating Token Index: 132

Text #13

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4407. Min Act: -0.0000

Data Index: 1546500 (Python Code)

Max Activating Token Index: 915

Text #14

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.6010. Min Act: -0.0000

Data Index: 1502677 (Python Code)

Max Activating Token Index: 471

Text #15

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.5270. Min Act: -0.0000

Data Index: 1531512 (Python Code)

Max Activating Token Index: 623

Text #16

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.3919. Min Act: -0.0000

Data Index: 1400159 (Python Code)

Max Activating Token Index: 893

Text #17

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4874. Min Act: -0.0000

Data Index: 1414199 (Python Code)

Max Activating Token Index: 44

Text #18

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4052. Min Act: -0.0000

Data Index: 1437802 (Python Code)

Max Activating Token Index: 163

Text #19

Max Range: 1.6625. Min Range: -1.6625

Max Act: 1.4841. Min Act: -0.0000

Data Index: 1559373 (Python Code)

Max Activating Token Index: 322