Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 815 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.8331. Min Act: -0.1700

Data Index: 1618520 (Python Code)

Max Activating Token Index: 997

Text #1

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.7026. Min Act: -0.1700

Data Index: 149468 (C4 (Web Text))

Max Activating Token Index: 446

Text #2

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.6922. Min Act: -0.1700

Data Index: 1423722 (Python Code)

Max Activating Token Index: 900

Text #3

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5966. Min Act: -0.1700

Data Index: 1465002 (Python Code)

Max Activating Token Index: 497

Text #4

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5673. Min Act: -0.1700

Data Index: 1496000 (Python Code)

Max Activating Token Index: 203

Text #5

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5055. Min Act: -0.1700

Data Index: 1431999 (Python Code)

Max Activating Token Index: 133

Text #6

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4708. Min Act: -0.1700

Data Index: 166255 (C4 (Web Text))

Max Activating Token Index: 34

Text #7

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4527. Min Act: -0.1700

Data Index: 968968 (C4 (Web Text))

Max Activating Token Index: 209

Text #8

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4571. Min Act: -0.1700

Data Index: 1202123 (C4 (Web Text))

Max Activating Token Index: 455

Text #9

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4549. Min Act: -0.1700

Data Index: 654345 (C4 (Web Text))

Max Activating Token Index: 206

Text #10

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4569. Min Act: -0.1700

Data Index: 633406 (C4 (Web Text))

Max Activating Token Index: 528

Text #11

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4393. Min Act: -0.1700

Data Index: 446222 (C4 (Web Text))

Max Activating Token Index: 753

Text #12

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4398. Min Act: -0.1700

Data Index: 661645 (C4 (Web Text))

Max Activating Token Index: 187

Text #13

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.3928. Min Act: -0.1700

Data Index: 1584072 (Python Code)

Max Activating Token Index: 515

Text #14

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.3529. Min Act: -0.1700

Data Index: 369983 (C4 (Web Text))

Max Activating Token Index: 310

Text #15

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.3565. Min Act: -0.1700

Data Index: 1456889 (Python Code)

Max Activating Token Index: 833

Text #16

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.3611. Min Act: -0.1700

Data Index: 969807 (C4 (Web Text))

Max Activating Token Index: 982

Text #17

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.3378. Min Act: -0.1700

Data Index: 1573556 (Python Code)

Max Activating Token Index: 821

Text #18

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.3200. Min Act: -0.1700

Data Index: 726090 (C4 (Web Text))

Max Activating Token Index: 76

Text #19

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.2989. Min Act: -0.1700

Data Index: 1627579 (Python Code)

Max Activating Token Index: 422