Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 579 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.3327. Min Act: -0.1700

Data Index: 1106311 (C4 (Web Text))

Max Activating Token Index: 474

Text #1

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.3349. Min Act: -0.1700

Data Index: 974065 (C4 (Web Text))

Max Activating Token Index: 304

Text #2

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2949. Min Act: -0.1700

Data Index: 1609703 (Python Code)

Max Activating Token Index: 378

Text #3

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2979. Min Act: -0.1700

Data Index: 203991 (C4 (Web Text))

Max Activating Token Index: 936

Text #4

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.3209. Min Act: -0.1700

Data Index: 167490 (C4 (Web Text))

Max Activating Token Index: 578

Text #5

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2796. Min Act: -0.1700

Data Index: 1461335 (Python Code)

Max Activating Token Index: 551

Text #6

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2732. Min Act: -0.1700

Data Index: 287373 (C4 (Web Text))

Max Activating Token Index: 89

Text #7

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2764. Min Act: -0.1700

Data Index: 486903 (C4 (Web Text))

Max Activating Token Index: 82

Text #8

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2449. Min Act: -0.1700

Data Index: 787118 (C4 (Web Text))

Max Activating Token Index: 410

Text #9

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2514. Min Act: -0.1700

Data Index: 1082282 (C4 (Web Text))

Max Activating Token Index: 421

Text #10

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2401. Min Act: -0.1700

Data Index: 459289 (C4 (Web Text))

Max Activating Token Index: 128

Text #11

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2438. Min Act: -0.1700

Data Index: 432586 (C4 (Web Text))

Max Activating Token Index: 632

Text #12

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2285. Min Act: -0.1700

Data Index: 210661 (C4 (Web Text))

Max Activating Token Index: 860

Text #13

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2112. Min Act: -0.1700

Data Index: 1018853 (C4 (Web Text))

Max Activating Token Index: 970

Text #14

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2328. Min Act: -0.1700

Data Index: 1194357 (C4 (Web Text))

Max Activating Token Index: 498

Text #15

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.1923. Min Act: -0.1700

Data Index: 1326730 (C4 (Web Text))

Max Activating Token Index: 662

Text #16

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.1922. Min Act: -0.1700

Data Index: 202918 (C4 (Web Text))

Max Activating Token Index: 231

Text #17

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.2011. Min Act: -0.1700

Data Index: 141487 (C4 (Web Text))

Max Activating Token Index: 302

Text #18

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.1985. Min Act: -0.1700

Data Index: 976034 (C4 (Web Text))

Max Activating Token Index: 127

Text #19

Max Range: 3.3349. Min Range: -3.3349

Max Act: 3.1782. Min Act: -0.1700

Data Index: 954978 (C4 (Web Text))

Max Activating Token Index: 810