Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 958 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.8602. Min Act: -0.1700

Data Index: 499087 (C4 (Web Text))

Max Activating Token Index: 432

Text #1

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.8294. Min Act: -0.1700

Data Index: 257831 (C4 (Web Text))

Max Activating Token Index: 552

Text #2

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.7544. Min Act: -0.1700

Data Index: 952721 (C4 (Web Text))

Max Activating Token Index: 180

Text #3

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.7282. Min Act: -0.1700

Data Index: 1141129 (C4 (Web Text))

Max Activating Token Index: 605

Text #4

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6633. Min Act: -0.1700

Data Index: 596651 (C4 (Web Text))

Max Activating Token Index: 336

Text #5

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6490. Min Act: -0.1699

Data Index: 463319 (C4 (Web Text))

Max Activating Token Index: 863

Text #6

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6613. Min Act: -0.1700

Data Index: 781317 (C4 (Web Text))

Max Activating Token Index: 889

Text #7

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6256. Min Act: -0.1700

Data Index: 105138 (C4 (Web Text))

Max Activating Token Index: 13

Text #8

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6097. Min Act: -0.1700

Data Index: 1037572 (C4 (Web Text))

Max Activating Token Index: 45

Text #9

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6289. Min Act: -0.1700

Data Index: 475099 (C4 (Web Text))

Max Activating Token Index: 763

Text #10

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6248. Min Act: -0.1700

Data Index: 1225496 (C4 (Web Text))

Max Activating Token Index: 406

Text #11

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6116. Min Act: -0.1700

Data Index: 990388 (C4 (Web Text))

Max Activating Token Index: 214

Text #12

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6198. Min Act: -0.1700

Data Index: 1020260 (C4 (Web Text))

Max Activating Token Index: 652

Text #13

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6033. Min Act: -0.1699

Data Index: 674145 (C4 (Web Text))

Max Activating Token Index: 916

Text #14

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.6139. Min Act: -0.1700

Data Index: 44419 (C4 (Web Text))

Max Activating Token Index: 538

Text #15

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.5918. Min Act: -0.1700

Data Index: 338308 (C4 (Web Text))

Max Activating Token Index: 882

Text #16

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.5852. Min Act: -0.1700

Data Index: 253993 (C4 (Web Text))

Max Activating Token Index: 410

Text #17

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.5977. Min Act: -0.1700

Data Index: 371750 (C4 (Web Text))

Max Activating Token Index: 461

Text #18

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.5838. Min Act: -0.1700

Data Index: 546725 (C4 (Web Text))

Max Activating Token Index: 656

Text #19

Max Range: 3.8602. Min Range: -3.8602

Max Act: 3.5900. Min Act: -0.1700

Data Index: 834360 (C4 (Web Text))

Max Activating Token Index: 831