Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1719 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.7239. Min Act: -0.1700

Data Index: 425612 (C4 (Web Text))

Max Activating Token Index: 955

Text #1

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.7114. Min Act: -0.1333

Data Index: 203065 (C4 (Web Text))

Max Activating Token Index: 546

Text #2

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.6552. Min Act: -0.1699

Data Index: 1055250 (C4 (Web Text))

Max Activating Token Index: 478

Text #3

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.5815. Min Act: -0.1700

Data Index: 488174 (C4 (Web Text))

Max Activating Token Index: 1001

Text #4

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.5254. Min Act: -0.1700

Data Index: 272240 (C4 (Web Text))

Max Activating Token Index: 119

Text #5

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4935. Min Act: -0.1563

Data Index: 897778 (C4 (Web Text))

Max Activating Token Index: 955

Text #6

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4533. Min Act: -0.1700

Data Index: 1065530 (C4 (Web Text))

Max Activating Token Index: 370

Text #7

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4427. Min Act: -0.1699

Data Index: 1105329 (C4 (Web Text))

Max Activating Token Index: 779

Text #8

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4502. Min Act: -0.1600

Data Index: 962651 (C4 (Web Text))

Max Activating Token Index: 270

Text #9

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4300. Min Act: -0.1700

Data Index: 1350447 (C4 (Web Text))

Max Activating Token Index: 118

Text #10

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4252. Min Act: -0.1642

Data Index: 342163 (C4 (Web Text))

Max Activating Token Index: 210

Text #11

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4294. Min Act: -0.1700

Data Index: 781592 (C4 (Web Text))

Max Activating Token Index: 987

Text #12

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.4282. Min Act: -0.1700

Data Index: 246322 (C4 (Web Text))

Max Activating Token Index: 871

Text #13

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.3862. Min Act: -0.1645

Data Index: 1028851 (C4 (Web Text))

Max Activating Token Index: 961

Text #14

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.3820. Min Act: -0.1700

Data Index: 1045302 (C4 (Web Text))

Max Activating Token Index: 610

Text #15

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.3923. Min Act: -0.1700

Data Index: 960565 (C4 (Web Text))

Max Activating Token Index: 313

Text #16

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.3887. Min Act: -0.1700

Data Index: 1279458 (C4 (Web Text))

Max Activating Token Index: 278

Text #17

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.3947. Min Act: -0.1698

Data Index: 507946 (C4 (Web Text))

Max Activating Token Index: 423

Text #18

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.3484. Min Act: -0.1698

Data Index: 808367 (C4 (Web Text))

Max Activating Token Index: 590

Text #19

Max Range: 3.7239. Min Range: -3.7239

Max Act: 3.3547. Min Act: -0.1640

Data Index: 1005751 (C4 (Web Text))

Max Activating Token Index: 983