Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 879 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.4884. Min Act: -0.1700

Data Index: 1239183 (C4 (Web Text))

Max Activating Token Index: 488

Text #1

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.4794. Min Act: -0.1700

Data Index: 258516 (C4 (Web Text))

Max Activating Token Index: 171

Text #2

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.4297. Min Act: -0.1700

Data Index: 337826 (C4 (Web Text))

Max Activating Token Index: 193

Text #3

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3841. Min Act: -0.1700

Data Index: 281812 (C4 (Web Text))

Max Activating Token Index: 787

Text #4

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3818. Min Act: -0.1700

Data Index: 397185 (C4 (Web Text))

Max Activating Token Index: 314

Text #5

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3742. Min Act: -0.1700

Data Index: 1003987 (C4 (Web Text))

Max Activating Token Index: 856

Text #6

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3660. Min Act: -0.1700

Data Index: 382096 (C4 (Web Text))

Max Activating Token Index: 101

Text #7

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.2998. Min Act: -0.1700

Data Index: 1351345 (C4 (Web Text))

Max Activating Token Index: 991

Text #8

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3522. Min Act: -0.1700

Data Index: 321912 (C4 (Web Text))

Max Activating Token Index: 157

Text #9

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3284. Min Act: -0.1700

Data Index: 763782 (C4 (Web Text))

Max Activating Token Index: 381

Text #10

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3403. Min Act: -0.1700

Data Index: 414835 (C4 (Web Text))

Max Activating Token Index: 46

Text #11

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3131. Min Act: -0.1700

Data Index: 577598 (C4 (Web Text))

Max Activating Token Index: 411

Text #12

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3183. Min Act: -0.1700

Data Index: 731880 (C4 (Web Text))

Max Activating Token Index: 383

Text #13

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3143. Min Act: -0.1700

Data Index: 110751 (C4 (Web Text))

Max Activating Token Index: 136

Text #14

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3326. Min Act: -0.1700

Data Index: 648103 (C4 (Web Text))

Max Activating Token Index: 696

Text #15

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.2999. Min Act: -0.1700

Data Index: 121810 (C4 (Web Text))

Max Activating Token Index: 1009

Text #16

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.3061. Min Act: -0.1700

Data Index: 548623 (C4 (Web Text))

Max Activating Token Index: 686

Text #17

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.2887. Min Act: -0.1700

Data Index: 907545 (C4 (Web Text))

Max Activating Token Index: 459

Text #18

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.2840. Min Act: -0.1700

Data Index: 1146236 (C4 (Web Text))

Max Activating Token Index: 1020

Text #19

Max Range: 4.4884. Min Range: -4.4884

Max Act: 4.2939. Min Act: -0.1700

Data Index: 538786 (C4 (Web Text))

Max Activating Token Index: 451