Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1480 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.3765. Min Range: -4.3765

Max Act: 4.3765. Min Act: -0.1700

Data Index: 996048 (C4 (Web Text))

Max Activating Token Index: 479

Text #1

Max Range: 4.3765. Min Range: -4.3765

Max Act: 4.2153. Min Act: -0.1700

Data Index: 352864 (C4 (Web Text))

Max Activating Token Index: 822

Text #2

Max Range: 4.3765. Min Range: -4.3765

Max Act: 4.1093. Min Act: -0.1700

Data Index: 1346139 (C4 (Web Text))

Max Activating Token Index: 568

Text #3

Max Range: 4.3765. Min Range: -4.3765

Max Act: 4.0630. Min Act: -0.1700

Data Index: 965316 (C4 (Web Text))

Max Activating Token Index: 759

Text #4

Max Range: 4.3765. Min Range: -4.3765

Max Act: 4.0596. Min Act: -0.1700

Data Index: 775486 (C4 (Web Text))

Max Activating Token Index: 196

Text #5

Max Range: 4.3765. Min Range: -4.3765

Max Act: 4.0526. Min Act: -0.1700

Data Index: 331714 (C4 (Web Text))

Max Activating Token Index: 605

Text #6

Max Range: 4.3765. Min Range: -4.3765

Max Act: 4.0286. Min Act: -0.1700

Data Index: 863963 (C4 (Web Text))

Max Activating Token Index: 26

Text #7

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.9126. Min Act: -0.1700

Data Index: 1119978 (C4 (Web Text))

Max Activating Token Index: 770

Text #8

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.8906. Min Act: -0.1700

Data Index: 249410 (C4 (Web Text))

Max Activating Token Index: 577

Text #9

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.7993. Min Act: -0.1700

Data Index: 60527 (C4 (Web Text))

Max Activating Token Index: 47

Text #10

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.7748. Min Act: -0.1700

Data Index: 1116379 (C4 (Web Text))

Max Activating Token Index: 497

Text #11

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.7478. Min Act: -0.1700

Data Index: 811905 (C4 (Web Text))

Max Activating Token Index: 1018

Text #12

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.7278. Min Act: -0.1700

Data Index: 1225877 (C4 (Web Text))

Max Activating Token Index: 124

Text #13

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.7229. Min Act: -0.1700

Data Index: 462208 (C4 (Web Text))

Max Activating Token Index: 830

Text #14

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.6988. Min Act: -0.1700

Data Index: 1375706 (Python Code)

Max Activating Token Index: 501

Text #15

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.6739. Min Act: -0.1700

Data Index: 88114 (C4 (Web Text))

Max Activating Token Index: 185

Text #16

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.6614. Min Act: -0.1700

Data Index: 416079 (C4 (Web Text))

Max Activating Token Index: 224

Text #17

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.6540. Min Act: -0.1700

Data Index: 1243685 (C4 (Web Text))

Max Activating Token Index: 70

Text #18

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.6441. Min Act: -0.1700

Data Index: 927299 (C4 (Web Text))

Max Activating Token Index: 269

Text #19

Max Range: 4.3765. Min Range: -4.3765

Max Act: 3.6655. Min Act: -0.1700

Data Index: 597897 (C4 (Web Text))

Max Activating Token Index: 871