Model: SoLU Model: 12 Layers, 6144 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 3384 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('solu-12l')

Click the toggle to see the full text!

Text #0

Max Range: 1.1166. Min Range: -1.1166

Max Act: 1.0955. Min Act: -0.0002

Data Index: 1400058 (Python Code)

Max Activating Token Index: 744

Text #1

Max Range: 1.1166. Min Range: -1.1166

Max Act: 1.1166. Min Act: -0.0002

Data Index: 1492522 (Python Code)

Max Activating Token Index: 160

Text #2

Max Range: 1.1166. Min Range: -1.1166

Max Act: 1.0809. Min Act: -0.0002

Data Index: 1428308 (Python Code)

Max Activating Token Index: 51

Text #3

Max Range: 1.1166. Min Range: -1.1166

Max Act: 1.0759. Min Act: -0.0002

Data Index: 1477342 (Python Code)

Max Activating Token Index: 249

Text #4

Max Range: 1.1166. Min Range: -1.1166

Max Act: 1.0608. Min Act: -0.0002

Data Index: 1542221 (Python Code)

Max Activating Token Index: 748

Text #5

Max Range: 1.1166. Min Range: -1.1166

Max Act: 1.0329. Min Act: -0.0002

Data Index: 1498762 (Python Code)

Max Activating Token Index: 100

Text #6

Max Range: 1.1166. Min Range: -1.1166

Max Act: 1.0122. Min Act: -0.0002

Data Index: 1403217 (Python Code)

Max Activating Token Index: 675

Text #7

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9712. Min Act: -0.0002

Data Index: 1372942 (Python Code)

Max Activating Token Index: 129

Text #8

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9532. Min Act: -0.0002

Data Index: 1520183 (Python Code)

Max Activating Token Index: 212

Text #9

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9510. Min Act: -0.0002

Data Index: 1577906 (Python Code)

Max Activating Token Index: 272

Text #10

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9818. Min Act: -0.0002

Data Index: 1376684 (Python Code)

Max Activating Token Index: 286

Text #11

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9702. Min Act: -0.0002

Data Index: 1613788 (Python Code)

Max Activating Token Index: 796

Text #12

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9526. Min Act: -0.0002

Data Index: 1451494 (Python Code)

Max Activating Token Index: 273

Text #13

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9470. Min Act: -0.0002

Data Index: 1522803 (Python Code)

Max Activating Token Index: 339

Text #14

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9370. Min Act: -0.0002

Data Index: 1371686 (Python Code)

Max Activating Token Index: 339

Text #15

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9469. Min Act: -0.0002

Data Index: 1415575 (Python Code)

Max Activating Token Index: 29

Text #16

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9434. Min Act: -0.0002

Data Index: 1415550 (Python Code)

Max Activating Token Index: 970

Text #17

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9614. Min Act: -0.0002

Data Index: 1594545 (Python Code)

Max Activating Token Index: 293

Text #18

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9538. Min Act: -0.0002

Data Index: 1445458 (Python Code)

Max Activating Token Index: 453

Text #19

Max Range: 1.1166. Min Range: -1.1166

Max Act: 0.9466. Min Act: -0.0002

Data Index: 1536434 (Python Code)

Max Activating Token Index: 802