Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1258 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.7472. Min Act: -0.1700

Data Index: 1141272 (C4 (Web Text))

Max Activating Token Index: 230

Text #1

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.6222. Min Act: -0.1700

Data Index: 808775 (C4 (Web Text))

Max Activating Token Index: 124

Text #2

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.5755. Min Act: -0.1700

Data Index: 1164768 (C4 (Web Text))

Max Activating Token Index: 367

Text #3

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.5695. Min Act: -0.1700

Data Index: 1321181 (C4 (Web Text))

Max Activating Token Index: 263

Text #4

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.4287. Min Act: -0.1700

Data Index: 659795 (C4 (Web Text))

Max Activating Token Index: 187

Text #5

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.2055. Min Act: -0.1700

Data Index: 1650765 (Python Code)

Max Activating Token Index: 805

Text #6

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.2475. Min Act: -0.1700

Data Index: 1394136 (Python Code)

Max Activating Token Index: 790

Text #7

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.1879. Min Act: -0.1700

Data Index: 443346 (C4 (Web Text))

Max Activating Token Index: 534

Text #8

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.1940. Min Act: -0.1700

Data Index: 916671 (C4 (Web Text))

Max Activating Token Index: 679

Text #9

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.1823. Min Act: -0.1700

Data Index: 1150683 (C4 (Web Text))

Max Activating Token Index: 550

Text #10

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.1456. Min Act: -0.1700

Data Index: 1612765 (Python Code)

Max Activating Token Index: 120

Text #11

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.0898. Min Act: -0.1700

Data Index: 1510945 (Python Code)

Max Activating Token Index: 567

Text #12

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.1123. Min Act: -0.1700

Data Index: 1492098 (Python Code)

Max Activating Token Index: 943

Text #13

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.0797. Min Act: -0.1700

Data Index: 1654110 (Python Code)

Max Activating Token Index: 462

Text #14

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.0883. Min Act: -0.1700

Data Index: 172947 (C4 (Web Text))

Max Activating Token Index: 601

Text #15

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.0554. Min Act: -0.1700

Data Index: 1439494 (Python Code)

Max Activating Token Index: 464

Text #16

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.0543. Min Act: -0.1700

Data Index: 1639375 (Python Code)

Max Activating Token Index: 690

Text #17

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.1159. Min Act: -0.1700

Data Index: 1592073 (Python Code)

Max Activating Token Index: 31

Text #18

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.0773. Min Act: -0.1700

Data Index: 1491519 (Python Code)

Max Activating Token Index: 815

Text #19

Max Range: 4.7472. Min Range: -4.7472

Max Act: 4.1051. Min Act: -0.1700

Data Index: 1423025 (Python Code)

Max Activating Token Index: 592