Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1434 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.7596. Min Act: -0.1700

Data Index: 1402666 (Python Code)

Max Activating Token Index: 376

Text #1

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.6126. Min Act: -0.1700

Data Index: 1598722 (Python Code)

Max Activating Token Index: 819

Text #2

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.5223. Min Act: -0.1700

Data Index: 1638872 (Python Code)

Max Activating Token Index: 228

Text #3

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.5252. Min Act: -0.1700

Data Index: 1456697 (Python Code)

Max Activating Token Index: 241

Text #4

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.5083. Min Act: -0.1700

Data Index: 1623706 (Python Code)

Max Activating Token Index: 607

Text #5

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4769. Min Act: -0.1700

Data Index: 1368479 (Python Code)

Max Activating Token Index: 252

Text #6

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4899. Min Act: -0.1700

Data Index: 1568572 (Python Code)

Max Activating Token Index: 11

Text #7

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4454. Min Act: -0.1700

Data Index: 1384121 (Python Code)

Max Activating Token Index: 999

Text #8

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4424. Min Act: -0.1700

Data Index: 1607148 (Python Code)

Max Activating Token Index: 405

Text #9

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4358. Min Act: -0.1700

Data Index: 1406080 (Python Code)

Max Activating Token Index: 216

Text #10

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4017. Min Act: -0.1700

Data Index: 1431903 (Python Code)

Max Activating Token Index: 246

Text #11

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4085. Min Act: -0.1700

Data Index: 1599052 (Python Code)

Max Activating Token Index: 496

Text #12

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3832. Min Act: -0.1700

Data Index: 1533949 (Python Code)

Max Activating Token Index: 918

Text #13

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3896. Min Act: -0.1700

Data Index: 1366973 (Python Code)

Max Activating Token Index: 668

Text #14

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3873. Min Act: -0.1700

Data Index: 1620283 (Python Code)

Max Activating Token Index: 422

Text #15

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3433. Min Act: -0.1700

Data Index: 1453867 (Python Code)

Max Activating Token Index: 536

Text #16

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3431. Min Act: -0.1700

Data Index: 1440126 (Python Code)

Max Activating Token Index: 781

Text #17

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3835. Min Act: -0.1700

Data Index: 1588980 (Python Code)

Max Activating Token Index: 920

Text #18

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3262. Min Act: -0.1700

Data Index: 1440356 (Python Code)

Max Activating Token Index: 820

Text #19

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.3733. Min Act: -0.1700

Data Index: 1547426 (Python Code)

Max Activating Token Index: 350