Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1739 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.3879. Min Act: -0.1700

Data Index: 847934 (C4 (Web Text))

Max Activating Token Index: 294

Text #1

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.2860. Min Act: -0.1700

Data Index: 988318 (C4 (Web Text))

Max Activating Token Index: 817

Text #2

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.2909. Min Act: -0.1700

Data Index: 907811 (C4 (Web Text))

Max Activating Token Index: 289

Text #3

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.2480. Min Act: -0.1700

Data Index: 1353998 (C4 (Web Text))

Max Activating Token Index: 655

Text #4

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.2501. Min Act: -0.1700

Data Index: 1025941 (C4 (Web Text))

Max Activating Token Index: 347

Text #5

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.2687. Min Act: -0.1700

Data Index: 1197452 (C4 (Web Text))

Max Activating Token Index: 506

Text #6

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1868. Min Act: -0.1700

Data Index: 811510 (C4 (Web Text))

Max Activating Token Index: 312

Text #7

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1846. Min Act: -0.1700

Data Index: 936842 (C4 (Web Text))

Max Activating Token Index: 458

Text #8

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1502. Min Act: -0.1700

Data Index: 624395 (C4 (Web Text))

Max Activating Token Index: 185

Text #9

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1667. Min Act: -0.1700

Data Index: 466666 (C4 (Web Text))

Max Activating Token Index: 372

Text #10

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1500. Min Act: -0.1700

Data Index: 1006612 (C4 (Web Text))

Max Activating Token Index: 641

Text #11

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1076. Min Act: -0.1700

Data Index: 1024570 (C4 (Web Text))

Max Activating Token Index: 1013

Text #12

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1472. Min Act: -0.1700

Data Index: 911338 (C4 (Web Text))

Max Activating Token Index: 7

Text #13

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1325. Min Act: -0.1700

Data Index: 1298264 (C4 (Web Text))

Max Activating Token Index: 456

Text #14

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.0666. Min Act: -0.1700

Data Index: 444143 (C4 (Web Text))

Max Activating Token Index: 1015

Text #15

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1161. Min Act: -0.1700

Data Index: 639096 (C4 (Web Text))

Max Activating Token Index: 35

Text #16

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.1168. Min Act: -0.1700

Data Index: 480551 (C4 (Web Text))

Max Activating Token Index: 489

Text #17

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.0980. Min Act: -0.1700

Data Index: 1168905 (C4 (Web Text))

Max Activating Token Index: 130

Text #18

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.0956. Min Act: -0.1700

Data Index: 562064 (C4 (Web Text))

Max Activating Token Index: 733

Text #19

Max Range: 4.3879. Min Range: -4.3879

Max Act: 4.0865. Min Act: -0.1700

Data Index: 527460 (C4 (Web Text))

Max Activating Token Index: 5