Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 582 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.6554. Min Act: -0.1700

Data Index: 146818 (C4 (Web Text))

Max Activating Token Index: 556

Text #1

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.5826. Min Act: -0.1700

Data Index: 327668 (C4 (Web Text))

Max Activating Token Index: 174

Text #2

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.5725. Min Act: -0.1700

Data Index: 295157 (C4 (Web Text))

Max Activating Token Index: 263

Text #3

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.5694. Min Act: -0.1700

Data Index: 255868 (C4 (Web Text))

Max Activating Token Index: 242

Text #4

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4893. Min Act: -0.1700

Data Index: 342134 (C4 (Web Text))

Max Activating Token Index: 350

Text #5

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4554. Min Act: -0.1700

Data Index: 575765 (C4 (Web Text))

Max Activating Token Index: 187

Text #6

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4918. Min Act: -0.1700

Data Index: 1063359 (C4 (Web Text))

Max Activating Token Index: 313

Text #7

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4845. Min Act: -0.1700

Data Index: 415728 (C4 (Web Text))

Max Activating Token Index: 168

Text #8

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4851. Min Act: -0.1700

Data Index: 774845 (C4 (Web Text))

Max Activating Token Index: 450

Text #9

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.5377. Min Act: -0.1700

Data Index: 166066 (C4 (Web Text))

Max Activating Token Index: 796

Text #10

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4791. Min Act: -0.1700

Data Index: 201426 (C4 (Web Text))

Max Activating Token Index: 513

Text #11

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.3722. Min Act: -0.1700

Data Index: 952205 (C4 (Web Text))

Max Activating Token Index: 939

Text #12

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4827. Min Act: -0.1700

Data Index: 544132 (C4 (Web Text))

Max Activating Token Index: 337

Text #13

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4526. Min Act: -0.1700

Data Index: 378580 (C4 (Web Text))

Max Activating Token Index: 394

Text #14

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4526. Min Act: -0.1700

Data Index: 1175622 (C4 (Web Text))

Max Activating Token Index: 207

Text #15

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4547. Min Act: -0.1700

Data Index: 798401 (C4 (Web Text))

Max Activating Token Index: 132

Text #16

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.3993. Min Act: -0.1700

Data Index: 714089 (C4 (Web Text))

Max Activating Token Index: 102

Text #17

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.3767. Min Act: -0.1700

Data Index: 1066465 (C4 (Web Text))

Max Activating Token Index: 999

Text #18

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4661. Min Act: -0.1700

Data Index: 846286 (C4 (Web Text))

Max Activating Token Index: 568

Text #19

Max Range: 4.6554. Min Range: -4.6554

Max Act: 4.4009. Min Act: -0.1700

Data Index: 980263 (C4 (Web Text))

Max Activating Token Index: 598