Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 328 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.0892. Min Range: -4.0892

Max Act: 4.0892. Min Act: -0.1700

Data Index: 1434224 (Python Code)

Max Activating Token Index: 400

Text #1

Max Range: 4.0892. Min Range: -4.0892

Max Act: 4.0015. Min Act: -0.1700

Data Index: 1608433 (Python Code)

Max Activating Token Index: 953

Text #2

Max Range: 4.0892. Min Range: -4.0892

Max Act: 4.0318. Min Act: -0.1666

Data Index: 1421093 (Python Code)

Max Activating Token Index: 131

Text #3

Max Range: 4.0892. Min Range: -4.0892

Max Act: 4.0106. Min Act: -0.1339

Data Index: 1409270 (Python Code)

Max Activating Token Index: 508

Text #4

Max Range: 4.0892. Min Range: -4.0892

Max Act: 4.0069. Min Act: -0.1697

Data Index: 1602319 (Python Code)

Max Activating Token Index: 901

Text #5

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.9992. Min Act: -0.1625

Data Index: 1495786 (Python Code)

Max Activating Token Index: 511

Text #6

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.9598. Min Act: -0.1681

Data Index: 698784 (C4 (Web Text))

Max Activating Token Index: 204

Text #7

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.9461. Min Act: -0.1553

Data Index: 1408073 (Python Code)

Max Activating Token Index: 879

Text #8

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.9664. Min Act: -0.1698

Data Index: 1544621 (Python Code)

Max Activating Token Index: 730

Text #9

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.9129. Min Act: -0.1700

Data Index: 1489043 (Python Code)

Max Activating Token Index: 370

Text #10

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8800. Min Act: -0.1655

Data Index: 1626929 (Python Code)

Max Activating Token Index: 38

Text #11

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8934. Min Act: -0.1692

Data Index: 1591127 (Python Code)

Max Activating Token Index: 352

Text #12

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.9052. Min Act: -0.1329

Data Index: 1526879 (Python Code)

Max Activating Token Index: 94

Text #13

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8919. Min Act: -0.1700

Data Index: 1575069 (Python Code)

Max Activating Token Index: 121

Text #14

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8907. Min Act: -0.1693

Data Index: 1447432 (Python Code)

Max Activating Token Index: 470

Text #15

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8815. Min Act: -0.1692

Data Index: 1442778 (Python Code)

Max Activating Token Index: 346

Text #16

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8938. Min Act: -0.1673

Data Index: 1365156 (Python Code)

Max Activating Token Index: 467

Text #17

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.9057. Min Act: -0.1697

Data Index: 1618882 (Python Code)

Max Activating Token Index: 589

Text #18

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8826. Min Act: -0.1329

Data Index: 1510228 (Python Code)

Max Activating Token Index: 167

Text #19

Max Range: 4.0892. Min Range: -4.0892

Max Act: 3.8806. Min Act: -0.1574

Data Index: 1378840 (Python Code)

Max Activating Token Index: 476