Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1348 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.4199. Min Act: -0.1700

Data Index: 834389 (C4 (Web Text))

Max Activating Token Index: 37

Text #1

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3633. Min Act: -0.1700

Data Index: 1792 (C4 (Web Text))

Max Activating Token Index: 154

Text #2

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3309. Min Act: -0.1700

Data Index: 1198960 (C4 (Web Text))

Max Activating Token Index: 149

Text #3

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3425. Min Act: -0.1700

Data Index: 562315 (C4 (Web Text))

Max Activating Token Index: 444

Text #4

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3523. Min Act: -0.1700

Data Index: 162748 (C4 (Web Text))

Max Activating Token Index: 1022

Text #5

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3370. Min Act: -0.1700

Data Index: 910728 (C4 (Web Text))

Max Activating Token Index: 409

Text #6

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3196. Min Act: -0.1700

Data Index: 1290129 (C4 (Web Text))

Max Activating Token Index: 255

Text #7

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3089. Min Act: -0.1700

Data Index: 396834 (C4 (Web Text))

Max Activating Token Index: 131

Text #8

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.3054. Min Act: -0.1700

Data Index: 1269013 (C4 (Web Text))

Max Activating Token Index: 1015

Text #9

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2883. Min Act: -0.1700

Data Index: 453613 (C4 (Web Text))

Max Activating Token Index: 451

Text #10

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2785. Min Act: -0.1700

Data Index: 188899 (C4 (Web Text))

Max Activating Token Index: 375

Text #11

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2582. Min Act: -0.1700

Data Index: 893134 (C4 (Web Text))

Max Activating Token Index: 665

Text #12

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2675. Min Act: -0.1700

Data Index: 736115 (C4 (Web Text))

Max Activating Token Index: 64

Text #13

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2577. Min Act: -0.1700

Data Index: 87452 (C4 (Web Text))

Max Activating Token Index: 173

Text #14

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2390. Min Act: -0.1700

Data Index: 844544 (C4 (Web Text))

Max Activating Token Index: 686

Text #15

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2323. Min Act: -0.1700

Data Index: 311588 (C4 (Web Text))

Max Activating Token Index: 52

Text #16

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2210. Min Act: -0.1700

Data Index: 338717 (C4 (Web Text))

Max Activating Token Index: 890

Text #17

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2282. Min Act: -0.1700

Data Index: 621676 (C4 (Web Text))

Max Activating Token Index: 339

Text #18

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2197. Min Act: -0.1700

Data Index: 1046440 (C4 (Web Text))

Max Activating Token Index: 397

Text #19

Max Range: 3.4199. Min Range: -3.4199

Max Act: 3.2189. Min Act: -0.1700

Data Index: 655082 (C4 (Web Text))

Max Activating Token Index: 220