Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1981 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.4936. Min Range: -3.4936

Max Act: 3.4936. Min Act: -0.1700

Data Index: 1072699 (C4 (Web Text))

Max Activating Token Index: 128

Text #1

Max Range: 3.4936. Min Range: -3.4936

Max Act: 3.2903. Min Act: -0.1700

Data Index: 1358937 (C4 (Web Text))

Max Activating Token Index: 133

Text #2

Max Range: 3.4936. Min Range: -3.4936

Max Act: 3.0781. Min Act: -0.1700

Data Index: 200342 (C4 (Web Text))

Max Activating Token Index: 200

Text #3

Max Range: 3.4936. Min Range: -3.4936

Max Act: 3.0724. Min Act: -0.1700

Data Index: 958819 (C4 (Web Text))

Max Activating Token Index: 818

Text #4

Max Range: 3.4936. Min Range: -3.4936

Max Act: 3.0325. Min Act: -0.1700

Data Index: 1100386 (C4 (Web Text))

Max Activating Token Index: 952

Text #5

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.9323. Min Act: -0.1700

Data Index: 372850 (C4 (Web Text))

Max Activating Token Index: 482

Text #6

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.8950. Min Act: -0.1700

Data Index: 99071 (C4 (Web Text))

Max Activating Token Index: 853

Text #7

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.8744. Min Act: -0.1700

Data Index: 497924 (C4 (Web Text))

Max Activating Token Index: 859

Text #8

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.8446. Min Act: -0.1700

Data Index: 1180862 (C4 (Web Text))

Max Activating Token Index: 386

Text #9

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.8207. Min Act: -0.1700

Data Index: 400323 (C4 (Web Text))

Max Activating Token Index: 410

Text #10

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.8090. Min Act: -0.1700

Data Index: 751810 (C4 (Web Text))

Max Activating Token Index: 148

Text #11

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7978. Min Act: -0.1700

Data Index: 1277678 (C4 (Web Text))

Max Activating Token Index: 936

Text #12

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7799. Min Act: -0.1700

Data Index: 379706 (C4 (Web Text))

Max Activating Token Index: 1020

Text #13

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7737. Min Act: -0.1700

Data Index: 425565 (C4 (Web Text))

Max Activating Token Index: 971

Text #14

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7672. Min Act: -0.1700

Data Index: 615812 (C4 (Web Text))

Max Activating Token Index: 936

Text #15

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7342. Min Act: -0.1700

Data Index: 1193467 (C4 (Web Text))

Max Activating Token Index: 861

Text #16

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7357. Min Act: -0.1700

Data Index: 810503 (C4 (Web Text))

Max Activating Token Index: 454

Text #17

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7139. Min Act: -0.1700

Data Index: 139839 (C4 (Web Text))

Max Activating Token Index: 303

Text #18

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7159. Min Act: -0.1700

Data Index: 321659 (C4 (Web Text))

Max Activating Token Index: 535

Text #19

Max Range: 3.4936. Min Range: -3.4936

Max Act: 2.7355. Min Act: -0.1700

Data Index: 693920 (C4 (Web Text))

Max Activating Token Index: 520