Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1976 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.0337. Min Range: -3.0337

Max Act: 3.0337. Min Act: -0.1700

Data Index: 212065 (C4 (Web Text))

Max Activating Token Index: 290

Text #1

Max Range: 3.0337. Min Range: -3.0337

Max Act: 3.0048. Min Act: -0.1700

Data Index: 1422514 (Python Code)

Max Activating Token Index: 707

Text #2

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9606. Min Act: -0.1700

Data Index: 1132349 (C4 (Web Text))

Max Activating Token Index: 442

Text #3

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9526. Min Act: -0.1700

Data Index: 1647098 (Python Code)

Max Activating Token Index: 612

Text #4

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9432. Min Act: -0.1700

Data Index: 1654080 (Python Code)

Max Activating Token Index: 360

Text #5

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9385. Min Act: -0.1700

Data Index: 912872 (C4 (Web Text))

Max Activating Token Index: 798

Text #6

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9516. Min Act: -0.1700

Data Index: 358253 (C4 (Web Text))

Max Activating Token Index: 722

Text #7

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9382. Min Act: -0.1700

Data Index: 949566 (C4 (Web Text))

Max Activating Token Index: 501

Text #8

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9336. Min Act: -0.1700

Data Index: 1119142 (C4 (Web Text))

Max Activating Token Index: 385

Text #9

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9340. Min Act: -0.1700

Data Index: 1213640 (C4 (Web Text))

Max Activating Token Index: 978

Text #10

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9257. Min Act: -0.1699

Data Index: 45185 (C4 (Web Text))

Max Activating Token Index: 145

Text #11

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.9170. Min Act: -0.1700

Data Index: 1573633 (Python Code)

Max Activating Token Index: 95

Text #12

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8801. Min Act: -0.1700

Data Index: 854585 (C4 (Web Text))

Max Activating Token Index: 889

Text #13

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8823. Min Act: -0.1700

Data Index: 1557902 (Python Code)

Max Activating Token Index: 43

Text #14

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8901. Min Act: -0.1700

Data Index: 1420727 (Python Code)

Max Activating Token Index: 226

Text #15

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8972. Min Act: -0.1700

Data Index: 1314944 (C4 (Web Text))

Max Activating Token Index: 832

Text #16

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8729. Min Act: -0.1700

Data Index: 1113734 (C4 (Web Text))

Max Activating Token Index: 731

Text #17

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8701. Min Act: -0.1700

Data Index: 1522506 (Python Code)

Max Activating Token Index: 569

Text #18

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8520. Min Act: -0.1700

Data Index: 40681 (C4 (Web Text))

Max Activating Token Index: 445

Text #19

Max Range: 3.0337. Min Range: -3.0337

Max Act: 2.8809. Min Act: -0.1700

Data Index: 405858 (C4 (Web Text))

Max Activating Token Index: 602