Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 234 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.6216. Min Act: -0.1700

Data Index: 625632 (C4 (Web Text))

Max Activating Token Index: 811

Text #1

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.6056. Min Act: -0.1700

Data Index: 130055 (C4 (Web Text))

Max Activating Token Index: 768

Text #2

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.6064. Min Act: -0.1700

Data Index: 73879 (C4 (Web Text))

Max Activating Token Index: 632

Text #3

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5865. Min Act: -0.1700

Data Index: 662256 (C4 (Web Text))

Max Activating Token Index: 887

Text #4

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5931. Min Act: -0.1700

Data Index: 809596 (C4 (Web Text))

Max Activating Token Index: 750

Text #5

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5730. Min Act: -0.1699

Data Index: 1123158 (C4 (Web Text))

Max Activating Token Index: 469

Text #6

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5860. Min Act: -0.1700

Data Index: 1192567 (C4 (Web Text))

Max Activating Token Index: 384

Text #7

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5683. Min Act: -0.1700

Data Index: 371918 (C4 (Web Text))

Max Activating Token Index: 261

Text #8

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5684. Min Act: -0.1700

Data Index: 560695 (C4 (Web Text))

Max Activating Token Index: 136

Text #9

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5514. Min Act: -0.1700

Data Index: 638548 (C4 (Web Text))

Max Activating Token Index: 415

Text #10

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5660. Min Act: -0.1700

Data Index: 127466 (C4 (Web Text))

Max Activating Token Index: 88

Text #11

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5601. Min Act: -0.1700

Data Index: 273092 (C4 (Web Text))

Max Activating Token Index: 269

Text #12

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5623. Min Act: -0.1700

Data Index: 426975 (C4 (Web Text))

Max Activating Token Index: 955

Text #13

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5251. Min Act: -0.1699

Data Index: 144772 (C4 (Web Text))

Max Activating Token Index: 272

Text #14

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5367. Min Act: -0.1700

Data Index: 702909 (C4 (Web Text))

Max Activating Token Index: 954

Text #15

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.5042. Min Act: -0.1700

Data Index: 1100716 (C4 (Web Text))

Max Activating Token Index: 544

Text #16

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.4929. Min Act: -0.1700

Data Index: 1095920 (C4 (Web Text))

Max Activating Token Index: 132

Text #17

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.4980. Min Act: -0.1700

Data Index: 745145 (C4 (Web Text))

Max Activating Token Index: 745

Text #18

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.4815. Min Act: -0.1700

Data Index: 863127 (C4 (Web Text))

Max Activating Token Index: 201

Text #19

Max Range: 3.6216. Min Range: -3.6216

Max Act: 3.4729. Min Act: -0.1700

Data Index: 43937 (C4 (Web Text))

Max Activating Token Index: 270