Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 943 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.2147. Min Act: -0.1700

Data Index: 793471 (C4 (Web Text))

Max Activating Token Index: 148

Text #1

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1942. Min Act: -0.1700

Data Index: 825643 (C4 (Web Text))

Max Activating Token Index: 871

Text #2

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1875. Min Act: -0.1700

Data Index: 751601 (C4 (Web Text))

Max Activating Token Index: 419

Text #3

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1597. Min Act: -0.1700

Data Index: 7212 (C4 (Web Text))

Max Activating Token Index: 822

Text #4

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1500. Min Act: -0.1700

Data Index: 634919 (C4 (Web Text))

Max Activating Token Index: 233

Text #5

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1307. Min Act: -0.1700

Data Index: 698082 (C4 (Web Text))

Max Activating Token Index: 383

Text #6

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1398. Min Act: -0.1700

Data Index: 293785 (C4 (Web Text))

Max Activating Token Index: 877

Text #7

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1397. Min Act: -0.1700

Data Index: 136600 (C4 (Web Text))

Max Activating Token Index: 526

Text #8

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1444. Min Act: -0.1700

Data Index: 429559 (C4 (Web Text))

Max Activating Token Index: 938

Text #9

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0889. Min Act: -0.1700

Data Index: 1061201 (C4 (Web Text))

Max Activating Token Index: 604

Text #10

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1001. Min Act: -0.1700

Data Index: 429615 (C4 (Web Text))

Max Activating Token Index: 70

Text #11

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.1121. Min Act: -0.1700

Data Index: 950613 (C4 (Web Text))

Max Activating Token Index: 70

Text #12

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0956. Min Act: -0.1700

Data Index: 865568 (C4 (Web Text))

Max Activating Token Index: 272

Text #13

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0905. Min Act: -0.1700

Data Index: 656283 (C4 (Web Text))

Max Activating Token Index: 736

Text #14

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0790. Min Act: -0.1700

Data Index: 849142 (C4 (Web Text))

Max Activating Token Index: 588

Text #15

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0702. Min Act: -0.1700

Data Index: 350282 (C4 (Web Text))

Max Activating Token Index: 466

Text #16

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0591. Min Act: -0.1700

Data Index: 106035 (C4 (Web Text))

Max Activating Token Index: 786

Text #17

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0446. Min Act: -0.1700

Data Index: 707825 (C4 (Web Text))

Max Activating Token Index: 772

Text #18

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0402. Min Act: -0.1700

Data Index: 351997 (C4 (Web Text))

Max Activating Token Index: 806

Text #19

Max Range: 3.2147. Min Range: -3.2147

Max Act: 3.0562. Min Act: -0.1700

Data Index: 120478 (C4 (Web Text))

Max Activating Token Index: 845