Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1289 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.9902. Min Act: -0.1700

Data Index: 1345504 (C4 (Web Text))

Max Activating Token Index: 402

Text #1

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.8485. Min Act: -0.1700

Data Index: 1277747 (C4 (Web Text))

Max Activating Token Index: 28

Text #2

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7986. Min Act: -0.1700

Data Index: 200475 (C4 (Web Text))

Max Activating Token Index: 230

Text #3

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7253. Min Act: -0.1700

Data Index: 117737 (C4 (Web Text))

Max Activating Token Index: 325

Text #4

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7149. Min Act: -0.1700

Data Index: 1284176 (C4 (Web Text))

Max Activating Token Index: 778

Text #5

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7204. Min Act: -0.1700

Data Index: 712985 (C4 (Web Text))

Max Activating Token Index: 536

Text #6

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7096. Min Act: -0.1700

Data Index: 1195958 (C4 (Web Text))

Max Activating Token Index: 864

Text #7

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7357. Min Act: -0.1700

Data Index: 1247483 (C4 (Web Text))

Max Activating Token Index: 871

Text #8

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7314. Min Act: -0.1700

Data Index: 1222560 (C4 (Web Text))

Max Activating Token Index: 68

Text #9

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7320. Min Act: -0.1700

Data Index: 260727 (C4 (Web Text))

Max Activating Token Index: 152

Text #10

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7146. Min Act: -0.1700

Data Index: 703840 (C4 (Web Text))

Max Activating Token Index: 941

Text #11

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.7011. Min Act: -0.1700

Data Index: 846105 (C4 (Web Text))

Max Activating Token Index: 358

Text #12

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6819. Min Act: -0.1700

Data Index: 727539 (C4 (Web Text))

Max Activating Token Index: 468

Text #13

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6813. Min Act: -0.1700

Data Index: 810696 (C4 (Web Text))

Max Activating Token Index: 942

Text #14

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6610. Min Act: -0.1700

Data Index: 885232 (C4 (Web Text))

Max Activating Token Index: 439

Text #15

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6847. Min Act: -0.1700

Data Index: 397756 (C4 (Web Text))

Max Activating Token Index: 652

Text #16

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6781. Min Act: -0.1700

Data Index: 135535 (C4 (Web Text))

Max Activating Token Index: 27

Text #17

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6706. Min Act: -0.1700

Data Index: 227269 (C4 (Web Text))

Max Activating Token Index: 868

Text #18

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6818. Min Act: -0.1700

Data Index: 925395 (C4 (Web Text))

Max Activating Token Index: 812

Text #19

Max Range: 3.9902. Min Range: -3.9902

Max Act: 3.6690. Min Act: -0.1700

Data Index: 1186651 (C4 (Web Text))

Max Activating Token Index: 590