Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1833 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.5684. Min Act: -0.1700

Data Index: 1317085 (C4 (Web Text))

Max Activating Token Index: 20

Text #1

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.4715. Min Act: -0.1217

Data Index: 1119879 (C4 (Web Text))

Max Activating Token Index: 90

Text #2

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.4725. Min Act: -0.1606

Data Index: 1532703 (Python Code)

Max Activating Token Index: 354

Text #3

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.4453. Min Act: -0.1660

Data Index: 1467224 (Python Code)

Max Activating Token Index: 184

Text #4

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.3493. Min Act: -0.1689

Data Index: 1498559 (Python Code)

Max Activating Token Index: 940

Text #5

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.3462. Min Act: -0.1003

Data Index: 1445376 (Python Code)

Max Activating Token Index: 194

Text #6

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.3084. Min Act: -0.0843

Data Index: 1633037 (Python Code)

Max Activating Token Index: 443

Text #7

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2893. Min Act: -0.1695

Data Index: 383991 (C4 (Web Text))

Max Activating Token Index: 143

Text #8

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2919. Min Act: -0.1700

Data Index: 1655757 (Python Code)

Max Activating Token Index: 921

Text #9

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2839. Min Act: -0.1700

Data Index: 361165 (C4 (Web Text))

Max Activating Token Index: 551

Text #10

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2839. Min Act: -0.1700

Data Index: 40927 (C4 (Web Text))

Max Activating Token Index: 520

Text #11

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2799. Min Act: -0.1689

Data Index: 1507030 (Python Code)

Max Activating Token Index: 643

Text #12

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2730. Min Act: -0.0936

Data Index: 1533605 (Python Code)

Max Activating Token Index: 371

Text #13

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2664. Min Act: -0.1700

Data Index: 1418784 (Python Code)

Max Activating Token Index: 905

Text #14

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2632. Min Act: -0.1699

Data Index: 1624103 (Python Code)

Max Activating Token Index: 582

Text #15

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2521. Min Act: -0.0843

Data Index: 1463286 (Python Code)

Max Activating Token Index: 706

Text #16

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2490. Min Act: -0.1587

Data Index: 1489584 (Python Code)

Max Activating Token Index: 77

Text #17

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2436. Min Act: -0.1700

Data Index: 1099050 (C4 (Web Text))

Max Activating Token Index: 551

Text #18

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2295. Min Act: -0.0843

Data Index: 1436506 (Python Code)

Max Activating Token Index: 847

Text #19

Max Range: 3.5684. Min Range: -3.5684

Max Act: 3.2361. Min Act: -0.1671

Data Index: 1473676 (Python Code)

Max Activating Token Index: 513