Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1758 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.8407. Min Act: -0.1700

Data Index: 218174 (C4 (Web Text))

Max Activating Token Index: 326

Text #1

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.7344. Min Act: -0.1700

Data Index: 1060363 (C4 (Web Text))

Max Activating Token Index: 93

Text #2

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6994. Min Act: -0.1700

Data Index: 1342344 (C4 (Web Text))

Max Activating Token Index: 693

Text #3

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.7013. Min Act: -0.1700

Data Index: 206970 (C4 (Web Text))

Max Activating Token Index: 728

Text #4

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.7039. Min Act: -0.1700

Data Index: 1097100 (C4 (Web Text))

Max Activating Token Index: 379

Text #5

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6781. Min Act: -0.1700

Data Index: 354688 (C4 (Web Text))

Max Activating Token Index: 356

Text #6

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6442. Min Act: -0.1700

Data Index: 629144 (C4 (Web Text))

Max Activating Token Index: 29

Text #7

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6648. Min Act: -0.1700

Data Index: 592907 (C4 (Web Text))

Max Activating Token Index: 488

Text #8

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6527. Min Act: -0.1700

Data Index: 795436 (C4 (Web Text))

Max Activating Token Index: 506

Text #9

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6372. Min Act: -0.1700

Data Index: 1098853 (C4 (Web Text))

Max Activating Token Index: 254

Text #10

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6220. Min Act: -0.1700

Data Index: 653457 (C4 (Web Text))

Max Activating Token Index: 395

Text #11

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6226. Min Act: -0.1700

Data Index: 1149100 (C4 (Web Text))

Max Activating Token Index: 467

Text #12

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6430. Min Act: -0.1700

Data Index: 615221 (C4 (Web Text))

Max Activating Token Index: 509

Text #13

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.6436. Min Act: -0.1700

Data Index: 147445 (C4 (Web Text))

Max Activating Token Index: 310

Text #14

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.5703. Min Act: -0.1700

Data Index: 1159989 (C4 (Web Text))

Max Activating Token Index: 659

Text #15

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.5714. Min Act: -0.1700

Data Index: 81035 (C4 (Web Text))

Max Activating Token Index: 677

Text #16

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.5529. Min Act: -0.1700

Data Index: 879349 (C4 (Web Text))

Max Activating Token Index: 750

Text #17

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.5767. Min Act: -0.1700

Data Index: 249041 (C4 (Web Text))

Max Activating Token Index: 372

Text #18

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.5609. Min Act: -0.1700

Data Index: 648276 (C4 (Web Text))

Max Activating Token Index: 209

Text #19

Max Range: 3.8407. Min Range: -3.8407

Max Act: 3.5813. Min Act: -0.1700

Data Index: 163176 (C4 (Web Text))

Max Activating Token Index: 147