Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1958 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.9983. Min Act: -0.1700

Data Index: 1424444 (Python Code)

Max Activating Token Index: 641

Text #1

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.9439. Min Act: -0.1699

Data Index: 1377621 (Python Code)

Max Activating Token Index: 454

Text #2

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8820. Min Act: -0.1700

Data Index: 1405843 (Python Code)

Max Activating Token Index: 395

Text #3

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8723. Min Act: -0.1700

Data Index: 629916 (C4 (Web Text))

Max Activating Token Index: 927

Text #4

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8438. Min Act: -0.1700

Data Index: 1647441 (Python Code)

Max Activating Token Index: 712

Text #5

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8685. Min Act: -0.1700

Data Index: 768177 (C4 (Web Text))

Max Activating Token Index: 490

Text #6

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8298. Min Act: -0.1700

Data Index: 1437655 (Python Code)

Max Activating Token Index: 142

Text #7

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8140. Min Act: -0.1700

Data Index: 1399102 (Python Code)

Max Activating Token Index: 711

Text #8

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8222. Min Act: -0.1700

Data Index: 329705 (C4 (Web Text))

Max Activating Token Index: 845

Text #9

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8332. Min Act: -0.1700

Data Index: 1171744 (C4 (Web Text))

Max Activating Token Index: 803

Text #10

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8026. Min Act: -0.1700

Data Index: 1501249 (Python Code)

Max Activating Token Index: 185

Text #11

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.8293. Min Act: -0.1700

Data Index: 1443553 (Python Code)

Max Activating Token Index: 719

Text #12

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7866. Min Act: -0.1700

Data Index: 1631782 (Python Code)

Max Activating Token Index: 481

Text #13

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7856. Min Act: -0.1700

Data Index: 155805 (C4 (Web Text))

Max Activating Token Index: 784

Text #14

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7801. Min Act: -0.1690

Data Index: 1561177 (Python Code)

Max Activating Token Index: 494

Text #15

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7952. Min Act: -0.1699

Data Index: 1558966 (Python Code)

Max Activating Token Index: 265

Text #16

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7962. Min Act: -0.1697

Data Index: 1498942 (Python Code)

Max Activating Token Index: 744

Text #17

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7781. Min Act: -0.1700

Data Index: 1442996 (Python Code)

Max Activating Token Index: 754

Text #18

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7500. Min Act: -0.1699

Data Index: 1488409 (Python Code)

Max Activating Token Index: 577

Text #19

Max Range: 2.9983. Min Range: -2.9983

Max Act: 2.7422. Min Act: -0.1700

Data Index: 1414560 (Python Code)

Max Activating Token Index: 193