Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 119 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 5.1277. Min Range: -5.1277

Max Act: 5.1277. Min Act: -0.1700

Data Index: 1209553 (C4 (Web Text))

Max Activating Token Index: 602

Text #1

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.8272. Min Act: -0.1700

Data Index: 649588 (C4 (Web Text))

Max Activating Token Index: 1018

Text #2

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.8039. Min Act: -0.1700

Data Index: 722133 (C4 (Web Text))

Max Activating Token Index: 629

Text #3

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.8450. Min Act: -0.1700

Data Index: 533818 (C4 (Web Text))

Max Activating Token Index: 250

Text #4

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.8162. Min Act: -0.1700

Data Index: 1613573 (Python Code)

Max Activating Token Index: 413

Text #5

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.8392. Min Act: -0.1700

Data Index: 348898 (C4 (Web Text))

Max Activating Token Index: 936

Text #6

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.8276. Min Act: -0.1700

Data Index: 649452 (C4 (Web Text))

Max Activating Token Index: 564

Text #7

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.7851. Min Act: -0.1700

Data Index: 1340493 (C4 (Web Text))

Max Activating Token Index: 741

Text #8

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.7856. Min Act: -0.1700

Data Index: 28971 (C4 (Web Text))

Max Activating Token Index: 919

Text #9

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.7285. Min Act: -0.1700

Data Index: 493863 (C4 (Web Text))

Max Activating Token Index: 1010

Text #10

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.7573. Min Act: -0.1700

Data Index: 546361 (C4 (Web Text))

Max Activating Token Index: 347

Text #11

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.7258. Min Act: -0.1700

Data Index: 619061 (C4 (Web Text))

Max Activating Token Index: 392

Text #12

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.7177. Min Act: -0.1700

Data Index: 173381 (C4 (Web Text))

Max Activating Token Index: 100

Text #13

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.6849. Min Act: -0.1700

Data Index: 416138 (C4 (Web Text))

Max Activating Token Index: 829

Text #14

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.7056. Min Act: -0.1700

Data Index: 1185425 (C4 (Web Text))

Max Activating Token Index: 712

Text #15

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.6633. Min Act: -0.1700

Data Index: 371096 (C4 (Web Text))

Max Activating Token Index: 923

Text #16

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.6779. Min Act: -0.1700

Data Index: 444119 (C4 (Web Text))

Max Activating Token Index: 874

Text #17

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.6721. Min Act: -0.1700

Data Index: 1063441 (C4 (Web Text))

Max Activating Token Index: 927

Text #18

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.6691. Min Act: -0.1700

Data Index: 236842 (C4 (Web Text))

Max Activating Token Index: 174

Text #19

Max Range: 5.1277. Min Range: -5.1277

Max Act: 4.6751. Min Act: -0.1700

Data Index: 893333 (C4 (Web Text))

Max Activating Token Index: 423