Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 463 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.8635. Min Act: -0.1700

Data Index: 1508378 (Python Code)

Max Activating Token Index: 283

Text #1

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.8339. Min Act: -0.1700

Data Index: 1593633 (Python Code)

Max Activating Token Index: 872

Text #2

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.8316. Min Act: -0.1700

Data Index: 1656113 (Python Code)

Max Activating Token Index: 755

Text #3

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.8189. Min Act: -0.1700

Data Index: 1565992 (Python Code)

Max Activating Token Index: 854

Text #4

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7979. Min Act: -0.1700

Data Index: 1580546 (Python Code)

Max Activating Token Index: 931

Text #5

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.8077. Min Act: -0.1700

Data Index: 1530260 (Python Code)

Max Activating Token Index: 193

Text #6

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7968. Min Act: -0.1700

Data Index: 1556875 (Python Code)

Max Activating Token Index: 857

Text #7

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7400. Min Act: -0.1700

Data Index: 1488972 (Python Code)

Max Activating Token Index: 474

Text #8

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7407. Min Act: -0.1700

Data Index: 1508477 (Python Code)

Max Activating Token Index: 435

Text #9

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7327. Min Act: -0.1700

Data Index: 1422725 (Python Code)

Max Activating Token Index: 169

Text #10

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7136. Min Act: -0.1700

Data Index: 1396912 (Python Code)

Max Activating Token Index: 504

Text #11

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.6880. Min Act: -0.1700

Data Index: 1409423 (Python Code)

Max Activating Token Index: 570

Text #12

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7180. Min Act: -0.1699

Data Index: 1577351 (Python Code)

Max Activating Token Index: 9

Text #13

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.6884. Min Act: -0.1700

Data Index: 1642370 (Python Code)

Max Activating Token Index: 293

Text #14

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.7065. Min Act: -0.1699

Data Index: 1617262 (Python Code)

Max Activating Token Index: 11

Text #15

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.6804. Min Act: -0.1700

Data Index: 1584918 (Python Code)

Max Activating Token Index: 589

Text #16

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.6764. Min Act: -0.1700

Data Index: 1615098 (Python Code)

Max Activating Token Index: 286

Text #17

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.6890. Min Act: -0.1700

Data Index: 1446400 (Python Code)

Max Activating Token Index: 828

Text #18

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.6671. Min Act: -0.1700

Data Index: 1549870 (Python Code)

Max Activating Token Index: 567

Text #19

Max Range: 3.8635. Min Range: -3.8635

Max Act: 3.6695. Min Act: -0.1700

Data Index: 1615180 (Python Code)

Max Activating Token Index: 335