Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1703 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.5219. Min Act: -0.1700

Data Index: 1631771 (Python Code)

Max Activating Token Index: 668

Text #1

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.4621. Min Act: -0.1700

Data Index: 1475876 (Python Code)

Max Activating Token Index: 514

Text #2

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.4639. Min Act: -0.1700

Data Index: 1467221 (Python Code)

Max Activating Token Index: 119

Text #3

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.4041. Min Act: -0.1700

Data Index: 1457537 (Python Code)

Max Activating Token Index: 71

Text #4

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.3452. Min Act: -0.1699

Data Index: 1432005 (Python Code)

Max Activating Token Index: 888

Text #5

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.3075. Min Act: -0.1699

Data Index: 1552969 (Python Code)

Max Activating Token Index: 781

Text #6

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.3129. Min Act: -0.1700

Data Index: 1598873 (Python Code)

Max Activating Token Index: 277

Text #7

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2922. Min Act: -0.1700

Data Index: 1564835 (Python Code)

Max Activating Token Index: 83

Text #8

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2978. Min Act: -0.1699

Data Index: 1456594 (Python Code)

Max Activating Token Index: 38

Text #9

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2984. Min Act: -0.1700

Data Index: 1593592 (Python Code)

Max Activating Token Index: 286

Text #10

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.3032. Min Act: -0.1700

Data Index: 1510497 (Python Code)

Max Activating Token Index: 871

Text #11

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2733. Min Act: -0.1700

Data Index: 1458558 (Python Code)

Max Activating Token Index: 133

Text #12

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2403. Min Act: -0.1700

Data Index: 1634599 (Python Code)

Max Activating Token Index: 882

Text #13

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2620. Min Act: -0.1700

Data Index: 1470481 (Python Code)

Max Activating Token Index: 241

Text #14

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2503. Min Act: -0.1700

Data Index: 1644932 (Python Code)

Max Activating Token Index: 78

Text #15

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2575. Min Act: -0.1700

Data Index: 1517914 (Python Code)

Max Activating Token Index: 50

Text #16

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2618. Min Act: -0.1700

Data Index: 1444002 (Python Code)

Max Activating Token Index: 76

Text #17

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2494. Min Act: -0.1700

Data Index: 1441124 (Python Code)

Max Activating Token Index: 269

Text #18

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2408. Min Act: -0.1700

Data Index: 1471801 (Python Code)

Max Activating Token Index: 570

Text #19

Max Range: 3.5219. Min Range: -3.5219

Max Act: 3.2386. Min Act: -0.1699

Data Index: 1495849 (Python Code)

Max Activating Token Index: 680