Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 666 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.2653. Min Range: -3.2653

Max Act: 3.2653. Min Act: -0.1700

Data Index: 951623 (C4 (Web Text))

Max Activating Token Index: 435

Text #1

Max Range: 3.2653. Min Range: -3.2653

Max Act: 3.0431. Min Act: -0.1700

Data Index: 1416918 (Python Code)

Max Activating Token Index: 650

Text #2

Max Range: 3.2653. Min Range: -3.2653

Max Act: 3.0437. Min Act: -0.1700

Data Index: 508504 (C4 (Web Text))

Max Activating Token Index: 701

Text #3

Max Range: 3.2653. Min Range: -3.2653

Max Act: 3.0176. Min Act: -0.1700

Data Index: 1542614 (Python Code)

Max Activating Token Index: 394

Text #4

Max Range: 3.2653. Min Range: -3.2653

Max Act: 3.0042. Min Act: -0.1700

Data Index: 1553159 (Python Code)

Max Activating Token Index: 176

Text #5

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.9955. Min Act: -0.1699

Data Index: 468237 (C4 (Web Text))

Max Activating Token Index: 545

Text #6

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.9462. Min Act: -0.1700

Data Index: 1341998 (C4 (Web Text))

Max Activating Token Index: 915

Text #7

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.9481. Min Act: -0.1700

Data Index: 438417 (C4 (Web Text))

Max Activating Token Index: 657

Text #8

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.9442. Min Act: -0.1700

Data Index: 1029486 (C4 (Web Text))

Max Activating Token Index: 55

Text #9

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.9670. Min Act: -0.1700

Data Index: 1011230 (C4 (Web Text))

Max Activating Token Index: 671

Text #10

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.9006. Min Act: -0.1700

Data Index: 566331 (C4 (Web Text))

Max Activating Token Index: 213

Text #11

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8937. Min Act: -0.1700

Data Index: 1219936 (C4 (Web Text))

Max Activating Token Index: 230

Text #12

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8721. Min Act: -0.1700

Data Index: 144699 (C4 (Web Text))

Max Activating Token Index: 417

Text #13

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8897. Min Act: -0.1700

Data Index: 191744 (C4 (Web Text))

Max Activating Token Index: 403

Text #14

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8846. Min Act: -0.1700

Data Index: 516879 (C4 (Web Text))

Max Activating Token Index: 412

Text #15

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8583. Min Act: -0.1700

Data Index: 1151554 (C4 (Web Text))

Max Activating Token Index: 709

Text #16

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8753. Min Act: -0.1700

Data Index: 808819 (C4 (Web Text))

Max Activating Token Index: 826

Text #17

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8508. Min Act: -0.1700

Data Index: 679174 (C4 (Web Text))

Max Activating Token Index: 817

Text #18

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8673. Min Act: -0.1697

Data Index: 1143467 (C4 (Web Text))

Max Activating Token Index: 603

Text #19

Max Range: 3.2653. Min Range: -3.2653

Max Act: 2.8446. Min Act: -0.1700

Data Index: 1118835 (C4 (Web Text))

Max Activating Token Index: 725