Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1689 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.3178. Min Act: -0.1147

Data Index: 994480 (C4 (Web Text))

Max Activating Token Index: 592

Text #1

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2888. Min Act: -0.1600

Data Index: 1305115 (C4 (Web Text))

Max Activating Token Index: 254

Text #2

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.3099. Min Act: -0.1700

Data Index: 98659 (C4 (Web Text))

Max Activating Token Index: 164

Text #3

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2814. Min Act: -0.1698

Data Index: 455586 (C4 (Web Text))

Max Activating Token Index: 461

Text #4

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2958. Min Act: -0.1699

Data Index: 1252155 (C4 (Web Text))

Max Activating Token Index: 347

Text #5

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.3113. Min Act: -0.1696

Data Index: 387666 (C4 (Web Text))

Max Activating Token Index: 136

Text #6

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2672. Min Act: -0.1587

Data Index: 845009 (C4 (Web Text))

Max Activating Token Index: 957

Text #7

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2531. Min Act: -0.1522

Data Index: 830769 (C4 (Web Text))

Max Activating Token Index: 402

Text #8

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2540. Min Act: -0.1632

Data Index: 200519 (C4 (Web Text))

Max Activating Token Index: 155

Text #9

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2735. Min Act: -0.1533

Data Index: 571546 (C4 (Web Text))

Max Activating Token Index: 485

Text #10

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2175. Min Act: -0.1696

Data Index: 273738 (C4 (Web Text))

Max Activating Token Index: 816

Text #11

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2308. Min Act: -0.1685

Data Index: 431821 (C4 (Web Text))

Max Activating Token Index: 785

Text #12

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2444. Min Act: -0.1576

Data Index: 154815 (C4 (Web Text))

Max Activating Token Index: 373

Text #13

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.1877. Min Act: -0.1602

Data Index: 1120440 (C4 (Web Text))

Max Activating Token Index: 532

Text #14

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.1949. Min Act: -0.1699

Data Index: 1066772 (C4 (Web Text))

Max Activating Token Index: 177

Text #15

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2004. Min Act: -0.1700

Data Index: 578564 (C4 (Web Text))

Max Activating Token Index: 560

Text #16

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.1947. Min Act: -0.1531

Data Index: 534884 (C4 (Web Text))

Max Activating Token Index: 681

Text #17

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.1980. Min Act: -0.1438

Data Index: 1045596 (C4 (Web Text))

Max Activating Token Index: 322

Text #18

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.2037. Min Act: -0.1697

Data Index: 906021 (C4 (Web Text))

Max Activating Token Index: 775

Text #19

Max Range: 3.3178. Min Range: -3.3178

Max Act: 3.1760. Min Act: -0.1691

Data Index: 232393 (C4 (Web Text))

Max Activating Token Index: 735