Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 883 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.6570. Min Act: -0.1700

Data Index: 718010 (C4 (Web Text))

Max Activating Token Index: 779

Text #1

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.6360. Min Act: -0.1700

Data Index: 638595 (C4 (Web Text))

Max Activating Token Index: 892

Text #2

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.6471. Min Act: -0.1700

Data Index: 680081 (C4 (Web Text))

Max Activating Token Index: 729

Text #3

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.5639. Min Act: -0.1699

Data Index: 412499 (C4 (Web Text))

Max Activating Token Index: 570

Text #4

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.5449. Min Act: -0.1700

Data Index: 1140793 (C4 (Web Text))

Max Activating Token Index: 322

Text #5

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.5367. Min Act: -0.1700

Data Index: 724798 (C4 (Web Text))

Max Activating Token Index: 555

Text #6

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4840. Min Act: -0.1700

Data Index: 70033 (C4 (Web Text))

Max Activating Token Index: 432

Text #7

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4931. Min Act: -0.1700

Data Index: 1221704 (C4 (Web Text))

Max Activating Token Index: 842

Text #8

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4862. Min Act: -0.1700

Data Index: 210141 (C4 (Web Text))

Max Activating Token Index: 18

Text #9

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4715. Min Act: -0.1700

Data Index: 269118 (C4 (Web Text))

Max Activating Token Index: 954

Text #10

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4753. Min Act: -0.1700

Data Index: 1248750 (C4 (Web Text))

Max Activating Token Index: 381

Text #11

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4641. Min Act: -0.1700

Data Index: 816653 (C4 (Web Text))

Max Activating Token Index: 56

Text #12

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4742. Min Act: -0.1700

Data Index: 961277 (C4 (Web Text))

Max Activating Token Index: 117

Text #13

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4646. Min Act: -0.1700

Data Index: 1086584 (C4 (Web Text))

Max Activating Token Index: 546

Text #14

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4576. Min Act: -0.1700

Data Index: 767813 (C4 (Web Text))

Max Activating Token Index: 332

Text #15

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4470. Min Act: -0.1700

Data Index: 383941 (C4 (Web Text))

Max Activating Token Index: 54

Text #16

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4590. Min Act: -0.1700

Data Index: 848601 (C4 (Web Text))

Max Activating Token Index: 476

Text #17

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4444. Min Act: -0.1700

Data Index: 507658 (C4 (Web Text))

Max Activating Token Index: 912

Text #18

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4374. Min Act: -0.1700

Data Index: 321795 (C4 (Web Text))

Max Activating Token Index: 199

Text #19

Max Range: 3.6570. Min Range: -3.6570

Max Act: 3.4405. Min Act: -0.1700

Data Index: 660883 (C4 (Web Text))

Max Activating Token Index: 424