Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1692 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.3238. Min Act: -0.1700

Data Index: 613141 (C4 (Web Text))

Max Activating Token Index: 258

Text #1

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.2947. Min Act: -0.1700

Data Index: 70459 (C4 (Web Text))

Max Activating Token Index: 315

Text #2

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.2797. Min Act: -0.1700

Data Index: 425143 (C4 (Web Text))

Max Activating Token Index: 794

Text #3

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.2377. Min Act: -0.1700

Data Index: 936910 (C4 (Web Text))

Max Activating Token Index: 456

Text #4

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1984. Min Act: -0.1700

Data Index: 757414 (C4 (Web Text))

Max Activating Token Index: 510

Text #5

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1573. Min Act: -0.1700

Data Index: 409736 (C4 (Web Text))

Max Activating Token Index: 776

Text #6

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1681. Min Act: -0.1700

Data Index: 783554 (C4 (Web Text))

Max Activating Token Index: 841

Text #7

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1779. Min Act: -0.1700

Data Index: 881365 (C4 (Web Text))

Max Activating Token Index: 869

Text #8

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1324. Min Act: -0.1700

Data Index: 968066 (C4 (Web Text))

Max Activating Token Index: 855

Text #9

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1330. Min Act: -0.1700

Data Index: 379650 (C4 (Web Text))

Max Activating Token Index: 891

Text #10

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1177. Min Act: -0.1700

Data Index: 1208655 (C4 (Web Text))

Max Activating Token Index: 67

Text #11

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.1230. Min Act: -0.1700

Data Index: 1185434 (C4 (Web Text))

Max Activating Token Index: 110

Text #12

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0983. Min Act: -0.1700

Data Index: 1089653 (C4 (Web Text))

Max Activating Token Index: 982

Text #13

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0931. Min Act: -0.1700

Data Index: 1162991 (C4 (Web Text))

Max Activating Token Index: 969

Text #14

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0907. Min Act: -0.1700

Data Index: 580394 (C4 (Web Text))

Max Activating Token Index: 795

Text #15

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0735. Min Act: -0.1700

Data Index: 613997 (C4 (Web Text))

Max Activating Token Index: 99

Text #16

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0687. Min Act: -0.1700

Data Index: 1307615 (C4 (Web Text))

Max Activating Token Index: 893

Text #17

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0752. Min Act: -0.1700

Data Index: 1201433 (C4 (Web Text))

Max Activating Token Index: 490

Text #18

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0816. Min Act: -0.1700

Data Index: 1122155 (C4 (Web Text))

Max Activating Token Index: 880

Text #19

Max Range: 3.3238. Min Range: -3.3238

Max Act: 3.0795. Min Act: -0.1700

Data Index: 1163814 (C4 (Web Text))

Max Activating Token Index: 118