Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1357 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0909. Min Act: -0.1700

Data Index: 653610 (C4 (Web Text))

Max Activating Token Index: 454

Text #1

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0623. Min Act: -0.1700

Data Index: 515880 (C4 (Web Text))

Max Activating Token Index: 240

Text #2

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0479. Min Act: -0.1700

Data Index: 180125 (C4 (Web Text))

Max Activating Token Index: 911

Text #3

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0462. Min Act: -0.1700

Data Index: 465329 (C4 (Web Text))

Max Activating Token Index: 185

Text #4

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0505. Min Act: -0.1700

Data Index: 1193302 (C4 (Web Text))

Max Activating Token Index: 820

Text #5

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0576. Min Act: -0.1700

Data Index: 656537 (C4 (Web Text))

Max Activating Token Index: 741

Text #6

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0445. Min Act: -0.1700

Data Index: 62896 (C4 (Web Text))

Max Activating Token Index: 666

Text #7

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0479. Min Act: -0.1700

Data Index: 1328371 (C4 (Web Text))

Max Activating Token Index: 547

Text #8

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0292. Min Act: -0.1700

Data Index: 839624 (C4 (Web Text))

Max Activating Token Index: 372

Text #9

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0430. Min Act: -0.1700

Data Index: 769818 (C4 (Web Text))

Max Activating Token Index: 189

Text #10

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0233. Min Act: -0.1700

Data Index: 1244445 (C4 (Web Text))

Max Activating Token Index: 491

Text #11

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0322. Min Act: -0.1700

Data Index: 24219 (C4 (Web Text))

Max Activating Token Index: 343

Text #12

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0405. Min Act: -0.1700

Data Index: 1313100 (C4 (Web Text))

Max Activating Token Index: 102

Text #13

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0440. Min Act: -0.1700

Data Index: 926988 (C4 (Web Text))

Max Activating Token Index: 160

Text #14

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0291. Min Act: -0.1700

Data Index: 321097 (C4 (Web Text))

Max Activating Token Index: 264

Text #15

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0419. Min Act: -0.1700

Data Index: 1032686 (C4 (Web Text))

Max Activating Token Index: 240

Text #16

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0182. Min Act: -0.1700

Data Index: 958872 (C4 (Web Text))

Max Activating Token Index: 220

Text #17

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0203. Min Act: -0.1700

Data Index: 980751 (C4 (Web Text))

Max Activating Token Index: 520

Text #18

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0194. Min Act: -0.1700

Data Index: 132740 (C4 (Web Text))

Max Activating Token Index: 121

Text #19

Max Range: 3.0909. Min Range: -3.0909

Max Act: 3.0081. Min Act: -0.1700

Data Index: 145532 (C4 (Web Text))

Max Activating Token Index: 644