Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 559 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.6176. Min Act: -0.1699

Data Index: 252530 (C4 (Web Text))

Max Activating Token Index: 290

Text #1

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.4472. Min Act: -0.1700

Data Index: 1148973 (C4 (Web Text))

Max Activating Token Index: 952

Text #2

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.4509. Min Act: -0.1699

Data Index: 680799 (C4 (Web Text))

Max Activating Token Index: 966

Text #3

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.4580. Min Act: -0.1696

Data Index: 823956 (C4 (Web Text))

Max Activating Token Index: 568

Text #4

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.4149. Min Act: -0.1599

Data Index: 264196 (C4 (Web Text))

Max Activating Token Index: 547

Text #5

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3411. Min Act: -0.1695

Data Index: 491686 (C4 (Web Text))

Max Activating Token Index: 163

Text #6

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3403. Min Act: -0.1700

Data Index: 223536 (C4 (Web Text))

Max Activating Token Index: 1001

Text #7

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3261. Min Act: -0.1699

Data Index: 111744 (C4 (Web Text))

Max Activating Token Index: 865

Text #8

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3138. Min Act: -0.1699

Data Index: 873122 (C4 (Web Text))

Max Activating Token Index: 402

Text #9

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3218. Min Act: -0.1697

Data Index: 1319620 (C4 (Web Text))

Max Activating Token Index: 721

Text #10

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3243. Min Act: -0.1699

Data Index: 1241312 (C4 (Web Text))

Max Activating Token Index: 97

Text #11

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3013. Min Act: -0.1693

Data Index: 1392137 (Python Code)

Max Activating Token Index: 118

Text #12

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.3065. Min Act: -0.1700

Data Index: 891014 (C4 (Web Text))

Max Activating Token Index: 600

Text #13

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.2821. Min Act: -0.1700

Data Index: 812258 (C4 (Web Text))

Max Activating Token Index: 279

Text #14

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.2926. Min Act: -0.1700

Data Index: 1090555 (C4 (Web Text))

Max Activating Token Index: 884

Text #15

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.2859. Min Act: -0.1699

Data Index: 57227 (C4 (Web Text))

Max Activating Token Index: 696

Text #16

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.2420. Min Act: -0.1700

Data Index: 626676 (C4 (Web Text))

Max Activating Token Index: 633

Text #17

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.2668. Min Act: -0.1700

Data Index: 681976 (C4 (Web Text))

Max Activating Token Index: 817

Text #18

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.2624. Min Act: -0.1700

Data Index: 141216 (C4 (Web Text))

Max Activating Token Index: 215

Text #19

Max Range: 3.6176. Min Range: -3.6176

Max Act: 3.2573. Min Act: -0.1700

Data Index: 255069 (C4 (Web Text))

Max Activating Token Index: 391