Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2047 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.8136. Min Act: -0.1700

Data Index: 20697 (C4 (Web Text))

Max Activating Token Index: 343

Text #1

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.7615. Min Act: -0.1700

Data Index: 476883 (C4 (Web Text))

Max Activating Token Index: 749

Text #2

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.7408. Min Act: -0.1700

Data Index: 708209 (C4 (Web Text))

Max Activating Token Index: 528

Text #3

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.7452. Min Act: -0.1700

Data Index: 287826 (C4 (Web Text))

Max Activating Token Index: 693

Text #4

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.7333. Min Act: -0.1700

Data Index: 1235998 (C4 (Web Text))

Max Activating Token Index: 320

Text #5

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6981. Min Act: -0.1700

Data Index: 1163974 (C4 (Web Text))

Max Activating Token Index: 912

Text #6

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6914. Min Act: -0.1700

Data Index: 422872 (C4 (Web Text))

Max Activating Token Index: 176

Text #7

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6947. Min Act: -0.1700

Data Index: 559119 (C4 (Web Text))

Max Activating Token Index: 707

Text #8

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6683. Min Act: -0.1700

Data Index: 1149521 (C4 (Web Text))

Max Activating Token Index: 476

Text #9

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6671. Min Act: -0.1700

Data Index: 1309942 (C4 (Web Text))

Max Activating Token Index: 289

Text #10

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6610. Min Act: -0.1700

Data Index: 542742 (C4 (Web Text))

Max Activating Token Index: 675

Text #11

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6344. Min Act: -0.1700

Data Index: 1012168 (C4 (Web Text))

Max Activating Token Index: 457

Text #12

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6408. Min Act: -0.1700

Data Index: 980182 (C4 (Web Text))

Max Activating Token Index: 6

Text #13

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6099. Min Act: -0.1700

Data Index: 1343079 (C4 (Web Text))

Max Activating Token Index: 1018

Text #14

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.5969. Min Act: -0.1700

Data Index: 88036 (C4 (Web Text))

Max Activating Token Index: 298

Text #15

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.6087. Min Act: -0.1700

Data Index: 745368 (C4 (Web Text))

Max Activating Token Index: 565

Text #16

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.5939. Min Act: -0.1700

Data Index: 678690 (C4 (Web Text))

Max Activating Token Index: 219

Text #17

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.5600. Min Act: -0.1700

Data Index: 343248 (C4 (Web Text))

Max Activating Token Index: 766

Text #18

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.5873. Min Act: -0.1700

Data Index: 878083 (C4 (Web Text))

Max Activating Token Index: 610

Text #19

Max Range: 3.8136. Min Range: -3.8136

Max Act: 3.5873. Min Act: -0.1700

Data Index: 1070124 (C4 (Web Text))

Max Activating Token Index: 908