Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1732 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.7180. Min Act: -0.1700

Data Index: 159275 (C4 (Web Text))

Max Activating Token Index: 790

Text #1

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.6208. Min Act: -0.1700

Data Index: 182629 (C4 (Web Text))

Max Activating Token Index: 338

Text #2

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.6069. Min Act: -0.1700

Data Index: 267316 (C4 (Web Text))

Max Activating Token Index: 961

Text #3

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.5970. Min Act: -0.1700

Data Index: 266340 (C4 (Web Text))

Max Activating Token Index: 108

Text #4

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.5918. Min Act: -0.1700

Data Index: 344265 (C4 (Web Text))

Max Activating Token Index: 780

Text #5

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.5652. Min Act: -0.1700

Data Index: 1259857 (C4 (Web Text))

Max Activating Token Index: 322

Text #6

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.5185. Min Act: -0.1700

Data Index: 284250 (C4 (Web Text))

Max Activating Token Index: 410

Text #7

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.5288. Min Act: -0.1700

Data Index: 1030510 (C4 (Web Text))

Max Activating Token Index: 431

Text #8

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.5131. Min Act: -0.1700

Data Index: 314304 (C4 (Web Text))

Max Activating Token Index: 801

Text #9

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.5103. Min Act: -0.1700

Data Index: 419674 (C4 (Web Text))

Max Activating Token Index: 924

Text #10

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4759. Min Act: -0.1700

Data Index: 496630 (C4 (Web Text))

Max Activating Token Index: 457

Text #11

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4938. Min Act: -0.1700

Data Index: 21194 (C4 (Web Text))

Max Activating Token Index: 961

Text #12

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4749. Min Act: -0.1696

Data Index: 383306 (C4 (Web Text))

Max Activating Token Index: 612

Text #13

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4737. Min Act: -0.1700

Data Index: 69715 (C4 (Web Text))

Max Activating Token Index: 1005

Text #14

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4594. Min Act: -0.1700

Data Index: 181417 (C4 (Web Text))

Max Activating Token Index: 599

Text #15

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4602. Min Act: -0.1699

Data Index: 940799 (C4 (Web Text))

Max Activating Token Index: 351

Text #16

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4743. Min Act: -0.1699

Data Index: 1069901 (C4 (Web Text))

Max Activating Token Index: 963

Text #17

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4483. Min Act: -0.1699

Data Index: 596765 (C4 (Web Text))

Max Activating Token Index: 62

Text #18

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4801. Min Act: -0.1700

Data Index: 926234 (C4 (Web Text))

Max Activating Token Index: 796

Text #19

Max Range: 4.7180. Min Range: -4.7180

Max Act: 4.4622. Min Act: -0.1700

Data Index: 217178 (C4 (Web Text))

Max Activating Token Index: 658