Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1705 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.9315. Min Act: -0.1700

Data Index: 210299 (C4 (Web Text))

Max Activating Token Index: 754

Text #1

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.8015. Min Act: -0.1700

Data Index: 267046 (C4 (Web Text))

Max Activating Token Index: 882

Text #2

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.8303. Min Act: -0.1700

Data Index: 1307731 (C4 (Web Text))

Max Activating Token Index: 452

Text #3

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7814. Min Act: -0.1700

Data Index: 829640 (C4 (Web Text))

Max Activating Token Index: 709

Text #4

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7888. Min Act: -0.1700

Data Index: 375504 (C4 (Web Text))

Max Activating Token Index: 539

Text #5

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.8295. Min Act: -0.1700

Data Index: 720401 (C4 (Web Text))

Max Activating Token Index: 935

Text #6

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.8178. Min Act: -0.1700

Data Index: 432367 (C4 (Web Text))

Max Activating Token Index: 985

Text #7

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.8182. Min Act: -0.1700

Data Index: 1085194 (C4 (Web Text))

Max Activating Token Index: 400

Text #8

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7591. Min Act: -0.1700

Data Index: 1272514 (C4 (Web Text))

Max Activating Token Index: 934

Text #9

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7350. Min Act: -0.1700

Data Index: 812595 (C4 (Web Text))

Max Activating Token Index: 545

Text #10

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7486. Min Act: -0.1700

Data Index: 205301 (C4 (Web Text))

Max Activating Token Index: 409

Text #11

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7890. Min Act: -0.1700

Data Index: 741574 (C4 (Web Text))

Max Activating Token Index: 37

Text #12

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7579. Min Act: -0.1700

Data Index: 803704 (C4 (Web Text))

Max Activating Token Index: 257

Text #13

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7692. Min Act: -0.1700

Data Index: 197250 (C4 (Web Text))

Max Activating Token Index: 145

Text #14

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7667. Min Act: -0.1700

Data Index: 452835 (C4 (Web Text))

Max Activating Token Index: 1007

Text #15

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7256. Min Act: -0.1700

Data Index: 814506 (C4 (Web Text))

Max Activating Token Index: 129

Text #16

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7006. Min Act: -0.1700

Data Index: 189101 (C4 (Web Text))

Max Activating Token Index: 993

Text #17

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.6889. Min Act: -0.1700

Data Index: 532112 (C4 (Web Text))

Max Activating Token Index: 824

Text #18

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.6680. Min Act: -0.1700

Data Index: 300063 (C4 (Web Text))

Max Activating Token Index: 979

Text #19

Max Range: 4.9315. Min Range: -4.9315

Max Act: 4.7029. Min Act: -0.1700

Data Index: 618604 (C4 (Web Text))

Max Activating Token Index: 1015