Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1403 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.2760. Min Range: -3.2760

Max Act: 3.2760. Min Act: -0.1695

Data Index: 1406873 (Python Code)

Max Activating Token Index: 813

Text #1

Max Range: 3.2760. Min Range: -3.2760

Max Act: 3.0447. Min Act: -0.1700

Data Index: 1446591 (Python Code)

Max Activating Token Index: 581

Text #2

Max Range: 3.2760. Min Range: -3.2760

Max Act: 3.0470. Min Act: -0.1700

Data Index: 486999 (C4 (Web Text))

Max Activating Token Index: 32

Text #3

Max Range: 3.2760. Min Range: -3.2760

Max Act: 3.0419. Min Act: -0.1700

Data Index: 1596502 (Python Code)

Max Activating Token Index: 861

Text #4

Max Range: 3.2760. Min Range: -3.2760

Max Act: 3.0223. Min Act: -0.1700

Data Index: 705167 (C4 (Web Text))

Max Activating Token Index: 875

Text #5

Max Range: 3.2760. Min Range: -3.2760

Max Act: 3.0196. Min Act: -0.1700

Data Index: 1525970 (Python Code)

Max Activating Token Index: 1010

Text #6

Max Range: 3.2760. Min Range: -3.2760

Max Act: 3.0048. Min Act: -0.1700

Data Index: 1620085 (Python Code)

Max Activating Token Index: 26

Text #7

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9687. Min Act: -0.1700

Data Index: 1506091 (Python Code)

Max Activating Token Index: 926

Text #8

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9737. Min Act: -0.1700

Data Index: 1470153 (Python Code)

Max Activating Token Index: 449

Text #9

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9526. Min Act: -0.1700

Data Index: 1496373 (Python Code)

Max Activating Token Index: 415

Text #10

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9377. Min Act: -0.1700

Data Index: 342986 (C4 (Web Text))

Max Activating Token Index: 905

Text #11

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9413. Min Act: -0.1700

Data Index: 1496206 (Python Code)

Max Activating Token Index: 662

Text #12

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9437. Min Act: -0.1700

Data Index: 1380611 (Python Code)

Max Activating Token Index: 734

Text #13

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9429. Min Act: -0.1700

Data Index: 1275439 (C4 (Web Text))

Max Activating Token Index: 1003

Text #14

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9453. Min Act: -0.1700

Data Index: 422522 (C4 (Web Text))

Max Activating Token Index: 519

Text #15

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9443. Min Act: -0.1700

Data Index: 503813 (C4 (Web Text))

Max Activating Token Index: 704

Text #16

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9368. Min Act: -0.1700

Data Index: 1334939 (C4 (Web Text))

Max Activating Token Index: 732

Text #17

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9142. Min Act: -0.1700

Data Index: 1427272 (Python Code)

Max Activating Token Index: 267

Text #18

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9182. Min Act: -0.1700

Data Index: 1064440 (C4 (Web Text))

Max Activating Token Index: 359

Text #19

Max Range: 3.2760. Min Range: -3.2760

Max Act: 2.9261. Min Act: -0.1700

Data Index: 1344727 (C4 (Web Text))

Max Activating Token Index: 155