Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1026 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.6870. Min Act: -0.1700

Data Index: 1098470 (C4 (Web Text))

Max Activating Token Index: 932

Text #1

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.6904. Min Act: -0.1700

Data Index: 484617 (C4 (Web Text))

Max Activating Token Index: 823

Text #2

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.5215. Min Act: -0.1700

Data Index: 277504 (C4 (Web Text))

Max Activating Token Index: 188

Text #3

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.5047. Min Act: -0.1700

Data Index: 1028693 (C4 (Web Text))

Max Activating Token Index: 355

Text #4

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.4825. Min Act: -0.1700

Data Index: 579751 (C4 (Web Text))

Max Activating Token Index: 425

Text #5

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.3824. Min Act: -0.1700

Data Index: 1209284 (C4 (Web Text))

Max Activating Token Index: 30

Text #6

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.3611. Min Act: -0.1700

Data Index: 563844 (C4 (Web Text))

Max Activating Token Index: 547

Text #7

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.3495. Min Act: -0.1700

Data Index: 687192 (C4 (Web Text))

Max Activating Token Index: 122

Text #8

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.3187. Min Act: -0.1700

Data Index: 574399 (C4 (Web Text))

Max Activating Token Index: 160

Text #9

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.2732. Min Act: -0.1700

Data Index: 236952 (C4 (Web Text))

Max Activating Token Index: 583

Text #10

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.2801. Min Act: -0.1700

Data Index: 130752 (C4 (Web Text))

Max Activating Token Index: 249

Text #11

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.2116. Min Act: -0.1700

Data Index: 274700 (C4 (Web Text))

Max Activating Token Index: 652

Text #12

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1306. Min Act: -0.1700

Data Index: 815731 (C4 (Web Text))

Max Activating Token Index: 238

Text #13

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1736. Min Act: -0.1700

Data Index: 837025 (C4 (Web Text))

Max Activating Token Index: 120

Text #14

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1503. Min Act: -0.1700

Data Index: 600346 (C4 (Web Text))

Max Activating Token Index: 283

Text #15

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1359. Min Act: -0.1700

Data Index: 1160421 (C4 (Web Text))

Max Activating Token Index: 393

Text #16

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1402. Min Act: -0.1700

Data Index: 499951 (C4 (Web Text))

Max Activating Token Index: 78

Text #17

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1089. Min Act: -0.1700

Data Index: 895343 (C4 (Web Text))

Max Activating Token Index: 797

Text #18

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1111. Min Act: -0.1700

Data Index: 1259045 (C4 (Web Text))

Max Activating Token Index: 381

Text #19

Max Range: 4.6904. Min Range: -4.6904

Max Act: 4.1303. Min Act: -0.1700

Data Index: 377377 (C4 (Web Text))

Max Activating Token Index: 585