Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 615 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.8413. Min Act: -0.1700

Data Index: 1381083 (Python Code)

Max Activating Token Index: 623

Text #1

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.7763. Min Act: -0.1700

Data Index: 1371812 (Python Code)

Max Activating Token Index: 987

Text #2

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.7141. Min Act: -0.1700

Data Index: 1545343 (Python Code)

Max Activating Token Index: 822

Text #3

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.7035. Min Act: -0.1700

Data Index: 1623819 (Python Code)

Max Activating Token Index: 661

Text #4

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.7302. Min Act: -0.1700

Data Index: 1526537 (Python Code)

Max Activating Token Index: 242

Text #5

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6947. Min Act: -0.1700

Data Index: 1434824 (Python Code)

Max Activating Token Index: 411

Text #6

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6951. Min Act: -0.1700

Data Index: 1523329 (Python Code)

Max Activating Token Index: 259

Text #7

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6659. Min Act: -0.1700

Data Index: 1441399 (Python Code)

Max Activating Token Index: 565

Text #8

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6176. Min Act: -0.1700

Data Index: 1533860 (Python Code)

Max Activating Token Index: 677

Text #9

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6226. Min Act: -0.1700

Data Index: 1505900 (Python Code)

Max Activating Token Index: 577

Text #10

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6452. Min Act: -0.1700

Data Index: 1362846 (Python Code)

Max Activating Token Index: 858

Text #11

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6324. Min Act: -0.1700

Data Index: 1463425 (Python Code)

Max Activating Token Index: 68

Text #12

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6093. Min Act: -0.1700

Data Index: 1641106 (Python Code)

Max Activating Token Index: 238

Text #13

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6132. Min Act: -0.1700

Data Index: 1436957 (Python Code)

Max Activating Token Index: 522

Text #14

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.6047. Min Act: -0.1700

Data Index: 1400564 (Python Code)

Max Activating Token Index: 848

Text #15

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.5978. Min Act: -0.1700

Data Index: 1491787 (Python Code)

Max Activating Token Index: 535

Text #16

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.5978. Min Act: -0.1700

Data Index: 1622921 (Python Code)

Max Activating Token Index: 693

Text #17

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.5644. Min Act: -0.1700

Data Index: 1637592 (Python Code)

Max Activating Token Index: 613

Text #18

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.5759. Min Act: -0.1700

Data Index: 1547224 (Python Code)

Max Activating Token Index: 907

Text #19

Max Range: 3.8413. Min Range: -3.8413

Max Act: 3.5725. Min Act: -0.1700

Data Index: 1395367 (Python Code)

Max Activating Token Index: 38