Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1436 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.4124. Min Act: -0.1700

Data Index: 1542416 (Python Code)

Max Activating Token Index: 531

Text #1

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.3733. Min Act: -0.1700

Data Index: 790082 (C4 (Web Text))

Max Activating Token Index: 279

Text #2

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.3782. Min Act: -0.1700

Data Index: 1401627 (Python Code)

Max Activating Token Index: 494

Text #3

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.3816. Min Act: -0.1700

Data Index: 1593897 (Python Code)

Max Activating Token Index: 843

Text #4

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.3601. Min Act: -0.1700

Data Index: 1585032 (Python Code)

Max Activating Token Index: 603

Text #5

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.3638. Min Act: -0.1700

Data Index: 1480936 (Python Code)

Max Activating Token Index: 376

Text #6

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.3214. Min Act: -0.1700

Data Index: 1556713 (Python Code)

Max Activating Token Index: 600

Text #7

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2966. Min Act: -0.1700

Data Index: 1030490 (C4 (Web Text))

Max Activating Token Index: 459

Text #8

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2786. Min Act: -0.1700

Data Index: 1437901 (Python Code)

Max Activating Token Index: 210

Text #9

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2760. Min Act: -0.1700

Data Index: 1537734 (Python Code)

Max Activating Token Index: 86

Text #10

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2745. Min Act: -0.1700

Data Index: 1447927 (Python Code)

Max Activating Token Index: 518

Text #11

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2835. Min Act: -0.1700

Data Index: 1518472 (Python Code)

Max Activating Token Index: 878

Text #12

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2825. Min Act: -0.1700

Data Index: 1433603 (Python Code)

Max Activating Token Index: 457

Text #13

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2500. Min Act: -0.1700

Data Index: 1467705 (Python Code)

Max Activating Token Index: 1021

Text #14

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2537. Min Act: -0.1700

Data Index: 1629377 (Python Code)

Max Activating Token Index: 8

Text #15

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2427. Min Act: -0.1700

Data Index: 1376297 (Python Code)

Max Activating Token Index: 319

Text #16

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2498. Min Act: -0.1700

Data Index: 1240867 (C4 (Web Text))

Max Activating Token Index: 48

Text #17

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2215. Min Act: -0.1700

Data Index: 989471 (C4 (Web Text))

Max Activating Token Index: 526

Text #18

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2403. Min Act: -0.1700

Data Index: 1095306 (C4 (Web Text))

Max Activating Token Index: 408

Text #19

Max Range: 3.4124. Min Range: -3.4124

Max Act: 3.2094. Min Act: -0.1700

Data Index: 1171504 (C4 (Web Text))

Max Activating Token Index: 569