Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 4284 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-12l')

Text #0

Max Range: 5.3766. Min Range: -5.3766

Max Act: 5.3766. Min Act: -0.0000

Data Index: 1443023 (Python Code)

Max Activating Token Index: 708

Text #1

Max Range: 5.3766. Min Range: -5.3766

Max Act: 5.3689. Min Act: -0.0000

Data Index: 1617033 (Python Code)

Max Activating Token Index: 94

Text #2

Max Range: 5.3766. Min Range: -5.3766

Max Act: 5.0182. Min Act: -0.0000

Data Index: 1418522 (Python Code)

Max Activating Token Index: 694

Text #3

Max Range: 5.3766. Min Range: -5.3766

Max Act: 5.1254. Min Act: -0.0000

Data Index: 1463551 (Python Code)

Max Activating Token Index: 585

Text #4

Max Range: 5.3766. Min Range: -5.3766

Max Act: 5.1938. Min Act: -0.0000

Data Index: 1361164 (Python Code)

Max Activating Token Index: 180

Text #5

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.9411. Min Act: -0.0000

Data Index: 1384139 (Python Code)

Max Activating Token Index: 964

Text #6

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.9189. Min Act: -0.0000

Data Index: 1464512 (Python Code)

Max Activating Token Index: 187

Text #7

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.5465. Min Act: -0.0000

Data Index: 1472475 (Python Code)

Max Activating Token Index: 854

Text #8

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.7146. Min Act: -0.0000

Data Index: 1436168 (Python Code)

Max Activating Token Index: 791

Text #9

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.9267. Min Act: -0.0000

Data Index: 1380982 (Python Code)

Max Activating Token Index: 492

Text #10

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.4124. Min Act: -0.0000

Data Index: 1494425 (Python Code)

Max Activating Token Index: 846

Text #11

Max Range: 5.3766. Min Range: -5.3766

Max Act: 5.0519. Min Act: -0.0000

Data Index: 1596922 (Python Code)

Max Activating Token Index: 907

Text #12

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.6107. Min Act: -0.0000

Data Index: 1426287 (Python Code)

Max Activating Token Index: 659

Text #13

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.6446. Min Act: -0.0000

Data Index: 1376056 (Python Code)

Max Activating Token Index: 347

Text #14

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.5798. Min Act: -0.0000

Data Index: 1512527 (Python Code)

Max Activating Token Index: 101

Text #15

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.3996. Min Act: -0.0000

Data Index: 1435593 (Python Code)

Max Activating Token Index: 482

Text #16

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.5404. Min Act: -0.0000

Data Index: 1607631 (Python Code)

Max Activating Token Index: 55

Text #17

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.3725. Min Act: -0.0000

Data Index: 1392030 (Python Code)

Max Activating Token Index: 63

Text #18

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.2233. Min Act: -0.0000

Data Index: 1562866 (Python Code)

Max Activating Token Index: 738

Text #19

Max Range: 5.3766. Min Range: -5.3766

Max Act: 4.3896. Min Act: -0.0000

Data Index: 1436297 (Python Code)

Max Activating Token Index: 638