Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 445 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.4320. Min Act: -0.1700

Data Index: 12121 (C4 (Web Text))

Max Activating Token Index: 866

Text #1

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.4234. Min Act: -0.1700

Data Index: 1077290 (C4 (Web Text))

Max Activating Token Index: 53

Text #2

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.4050. Min Act: -0.1700

Data Index: 252913 (C4 (Web Text))

Max Activating Token Index: 551

Text #3

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.3310. Min Act: -0.1700

Data Index: 434049 (C4 (Web Text))

Max Activating Token Index: 893

Text #4

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2984. Min Act: -0.1700

Data Index: 528027 (C4 (Web Text))

Max Activating Token Index: 267

Text #5

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.3195. Min Act: -0.1700

Data Index: 621119 (C4 (Web Text))

Max Activating Token Index: 602

Text #6

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2857. Min Act: -0.1700

Data Index: 765696 (C4 (Web Text))

Max Activating Token Index: 775

Text #7

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2887. Min Act: -0.1700

Data Index: 1339603 (C4 (Web Text))

Max Activating Token Index: 436

Text #8

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2954. Min Act: -0.1700

Data Index: 453200 (C4 (Web Text))

Max Activating Token Index: 770

Text #9

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2837. Min Act: -0.1700

Data Index: 1138727 (C4 (Web Text))

Max Activating Token Index: 400

Text #10

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.3036. Min Act: -0.1700

Data Index: 34394 (C4 (Web Text))

Max Activating Token Index: 965

Text #11

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2709. Min Act: -0.1700

Data Index: 213791 (C4 (Web Text))

Max Activating Token Index: 249

Text #12

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2967. Min Act: -0.1700

Data Index: 130184 (C4 (Web Text))

Max Activating Token Index: 1006

Text #13

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2567. Min Act: -0.1700

Data Index: 1058539 (C4 (Web Text))

Max Activating Token Index: 131

Text #14

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2719. Min Act: -0.1700

Data Index: 384296 (C4 (Web Text))

Max Activating Token Index: 651

Text #15

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2598. Min Act: -0.1700

Data Index: 997191 (C4 (Web Text))

Max Activating Token Index: 862

Text #16

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2561. Min Act: -0.1700

Data Index: 850449 (C4 (Web Text))

Max Activating Token Index: 40

Text #17

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2504. Min Act: -0.1700

Data Index: 79608 (C4 (Web Text))

Max Activating Token Index: 166

Text #18

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2583. Min Act: -0.1700

Data Index: 178083 (C4 (Web Text))

Max Activating Token Index: 630

Text #19

Max Range: 3.4320. Min Range: -3.4320

Max Act: 3.2393. Min Act: -0.1700

Data Index: 733552 (C4 (Web Text))

Max Activating Token Index: 674