Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 48 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.2855. Min Act: -0.1699

Data Index: 1037520 (C4 (Web Text))

Max Activating Token Index: 115

Text #1

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.2054. Min Act: -0.1700

Data Index: 219791 (C4 (Web Text))

Max Activating Token Index: 305

Text #2

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1752. Min Act: -0.1700

Data Index: 307043 (C4 (Web Text))

Max Activating Token Index: 507

Text #3

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1062. Min Act: -0.1700

Data Index: 1312213 (C4 (Web Text))

Max Activating Token Index: 815

Text #4

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1073. Min Act: -0.1700

Data Index: 1297859 (C4 (Web Text))

Max Activating Token Index: 675

Text #5

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0984. Min Act: -0.1700

Data Index: 1040806 (C4 (Web Text))

Max Activating Token Index: 23

Text #6

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0846. Min Act: -0.1699

Data Index: 1250530 (C4 (Web Text))

Max Activating Token Index: 682

Text #7

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0513. Min Act: -0.1700

Data Index: 13517 (C4 (Web Text))

Max Activating Token Index: 447

Text #8

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0571. Min Act: -0.1700

Data Index: 458164 (C4 (Web Text))

Max Activating Token Index: 461

Text #9

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0234. Min Act: -0.1700

Data Index: 1336387 (C4 (Web Text))

Max Activating Token Index: 507

Text #10

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0118. Min Act: -0.1700

Data Index: 305446 (C4 (Web Text))

Max Activating Token Index: 757

Text #11

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0044. Min Act: -0.1700

Data Index: 1425434 (Python Code)

Max Activating Token Index: 651

Text #12

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0113. Min Act: -0.1700

Data Index: 1261303 (C4 (Web Text))

Max Activating Token Index: 565

Text #13

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0143. Min Act: -0.1700

Data Index: 361003 (C4 (Web Text))

Max Activating Token Index: 865

Text #14

Max Range: 3.2855. Min Range: -3.2855

Max Act: 2.9910. Min Act: -0.1700

Data Index: 367358 (C4 (Web Text))

Max Activating Token Index: 999

Text #15

Max Range: 3.2855. Min Range: -3.2855

Max Act: 2.9807. Min Act: -0.1700

Data Index: 508009 (C4 (Web Text))

Max Activating Token Index: 757

Text #16

Max Range: 3.2855. Min Range: -3.2855

Max Act: 2.9743. Min Act: -0.1700

Data Index: 988495 (C4 (Web Text))

Max Activating Token Index: 20

Text #17

Max Range: 3.2855. Min Range: -3.2855

Max Act: 2.9663. Min Act: -0.1700

Data Index: 1062193 (C4 (Web Text))

Max Activating Token Index: 203

Text #18

Max Range: 3.2855. Min Range: -3.2855

Max Act: 2.9640. Min Act: -0.1700

Data Index: 922669 (C4 (Web Text))

Max Activating Token Index: 122

Text #19

Max Range: 3.2855. Min Range: -3.2855

Max Act: 2.9536. Min Act: -0.1699

Data Index: 721210 (C4 (Web Text))

Max Activating Token Index: 571