Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1361 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.6686. Min Act: -0.1700

Data Index: 849825 (C4 (Web Text))

Max Activating Token Index: 921

Text #1

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.5754. Min Act: -0.1700

Data Index: 1582236 (Python Code)

Max Activating Token Index: 360

Text #2

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.5745. Min Act: -0.1700

Data Index: 1567921 (Python Code)

Max Activating Token Index: 543

Text #3

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.5162. Min Act: -0.1700

Data Index: 704236 (C4 (Web Text))

Max Activating Token Index: 325

Text #4

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.5316. Min Act: -0.1700

Data Index: 312447 (C4 (Web Text))

Max Activating Token Index: 309

Text #5

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4790. Min Act: -0.1700

Data Index: 1394397 (Python Code)

Max Activating Token Index: 325

Text #6

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4873. Min Act: -0.1700

Data Index: 1515574 (Python Code)

Max Activating Token Index: 189

Text #7

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4899. Min Act: -0.1700

Data Index: 1114405 (C4 (Web Text))

Max Activating Token Index: 171

Text #8

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4584. Min Act: -0.1700

Data Index: 768259 (C4 (Web Text))

Max Activating Token Index: 173

Text #9

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4392. Min Act: -0.1700

Data Index: 99549 (C4 (Web Text))

Max Activating Token Index: 632

Text #10

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4369. Min Act: -0.1700

Data Index: 744536 (C4 (Web Text))

Max Activating Token Index: 39

Text #11

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4297. Min Act: -0.1700

Data Index: 1362457 (Python Code)

Max Activating Token Index: 626

Text #12

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4291. Min Act: -0.1700

Data Index: 1144547 (C4 (Web Text))

Max Activating Token Index: 642

Text #13

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4234. Min Act: -0.1700

Data Index: 1632662 (Python Code)

Max Activating Token Index: 951

Text #14

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4334. Min Act: -0.1700

Data Index: 1392852 (Python Code)

Max Activating Token Index: 661

Text #15

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.4132. Min Act: -0.1700

Data Index: 1494884 (Python Code)

Max Activating Token Index: 615

Text #16

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.3928. Min Act: -0.1700

Data Index: 302455 (C4 (Web Text))

Max Activating Token Index: 16

Text #17

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.3933. Min Act: -0.1700

Data Index: 1154236 (C4 (Web Text))

Max Activating Token Index: 361

Text #18

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.3973. Min Act: -0.1700

Data Index: 1363125 (Python Code)

Max Activating Token Index: 291

Text #19

Max Range: 3.6686. Min Range: -3.6686

Max Act: 3.3722. Min Act: -0.1700

Data Index: 1094416 (C4 (Web Text))

Max Activating Token Index: 199