Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1091 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.5734. Min Act: -0.1673

Data Index: 1010988 (C4 (Web Text))

Max Activating Token Index: 174

Text #1

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.4950. Min Act: -0.1700

Data Index: 75823 (C4 (Web Text))

Max Activating Token Index: 696

Text #2

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.4960. Min Act: -0.1700

Data Index: 358006 (C4 (Web Text))

Max Activating Token Index: 441

Text #3

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.4611. Min Act: -0.1700

Data Index: 824095 (C4 (Web Text))

Max Activating Token Index: 644

Text #4

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.4306. Min Act: -0.1692

Data Index: 560913 (C4 (Web Text))

Max Activating Token Index: 431

Text #5

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.4216. Min Act: -0.1581

Data Index: 1031221 (C4 (Web Text))

Max Activating Token Index: 393

Text #6

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3851. Min Act: -0.1652

Data Index: 684217 (C4 (Web Text))

Max Activating Token Index: 104

Text #7

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3924. Min Act: -0.1700

Data Index: 485491 (C4 (Web Text))

Max Activating Token Index: 68

Text #8

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3926. Min Act: -0.1700

Data Index: 306410 (C4 (Web Text))

Max Activating Token Index: 86

Text #9

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3768. Min Act: -0.1697

Data Index: 1094112 (C4 (Web Text))

Max Activating Token Index: 1017

Text #10

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3702. Min Act: -0.1700

Data Index: 1007406 (C4 (Web Text))

Max Activating Token Index: 770

Text #11

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3367. Min Act: -0.1700

Data Index: 123243 (C4 (Web Text))

Max Activating Token Index: 676

Text #12

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3347. Min Act: -0.1699

Data Index: 889227 (C4 (Web Text))

Max Activating Token Index: 736

Text #13

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3476. Min Act: -0.1700

Data Index: 497702 (C4 (Web Text))

Max Activating Token Index: 733

Text #14

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3319. Min Act: -0.1550

Data Index: 1270453 (C4 (Web Text))

Max Activating Token Index: 861

Text #15

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3258. Min Act: -0.1700

Data Index: 566883 (C4 (Web Text))

Max Activating Token Index: 274

Text #16

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3105. Min Act: -0.1692

Data Index: 330466 (C4 (Web Text))

Max Activating Token Index: 91

Text #17

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3194. Min Act: -0.1700

Data Index: 928234 (C4 (Web Text))

Max Activating Token Index: 892

Text #18

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3007. Min Act: -0.1699

Data Index: 39948 (C4 (Web Text))

Max Activating Token Index: 975

Text #19

Max Range: 3.5734. Min Range: -3.5734

Max Act: 3.3251. Min Act: -0.1637

Data Index: 806945 (C4 (Web Text))

Max Activating Token Index: 108