Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1398 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.1450. Min Range: -3.1450

Max Act: 3.1450. Min Act: -0.1699

Data Index: 324703 (C4 (Web Text))

Max Activating Token Index: 87

Text #1

Max Range: 3.1450. Min Range: -3.1450

Max Act: 3.0074. Min Act: -0.1700

Data Index: 153419 (C4 (Web Text))

Max Activating Token Index: 881

Text #2

Max Range: 3.1450. Min Range: -3.1450

Max Act: 3.0120. Min Act: -0.1700

Data Index: 739314 (C4 (Web Text))

Max Activating Token Index: 992

Text #3

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.9549. Min Act: -0.1700

Data Index: 223072 (C4 (Web Text))

Max Activating Token Index: 861

Text #4

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.9162. Min Act: -0.1700

Data Index: 735366 (C4 (Web Text))

Max Activating Token Index: 245

Text #5

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.9149. Min Act: -0.1700

Data Index: 996590 (C4 (Web Text))

Max Activating Token Index: 77

Text #6

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.9090. Min Act: -0.1700

Data Index: 557216 (C4 (Web Text))

Max Activating Token Index: 794

Text #7

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.9113. Min Act: -0.1700

Data Index: 169335 (C4 (Web Text))

Max Activating Token Index: 907

Text #8

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.8896. Min Act: -0.1700

Data Index: 368799 (C4 (Web Text))

Max Activating Token Index: 225

Text #9

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.8652. Min Act: -0.1700

Data Index: 130091 (C4 (Web Text))

Max Activating Token Index: 945

Text #10

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.8338. Min Act: -0.1700

Data Index: 88941 (C4 (Web Text))

Max Activating Token Index: 1022

Text #11

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.8125. Min Act: -0.1700

Data Index: 1189481 (C4 (Web Text))

Max Activating Token Index: 99

Text #12

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.8294. Min Act: -0.1700

Data Index: 1178632 (C4 (Web Text))

Max Activating Token Index: 153

Text #13

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.8050. Min Act: -0.1700

Data Index: 1344033 (C4 (Web Text))

Max Activating Token Index: 931

Text #14

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.7970. Min Act: -0.1700

Data Index: 173947 (C4 (Web Text))

Max Activating Token Index: 185

Text #15

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.8089. Min Act: -0.1700

Data Index: 271356 (C4 (Web Text))

Max Activating Token Index: 128

Text #16

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.7726. Min Act: -0.1700

Data Index: 432967 (C4 (Web Text))

Max Activating Token Index: 534

Text #17

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.7773. Min Act: -0.1700

Data Index: 446089 (C4 (Web Text))

Max Activating Token Index: 143

Text #18

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.7778. Min Act: -0.1700

Data Index: 1285620 (C4 (Web Text))

Max Activating Token Index: 493

Text #19

Max Range: 3.1450. Min Range: -3.1450

Max Act: 2.7852. Min Act: -0.1700

Data Index: 926845 (C4 (Web Text))

Max Activating Token Index: 1000