Model: GELU Model: 3 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 882 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('gelu-3l')

Text #0

Max Range: 4.0673. Min Range: -4.0673

Max Act: 4.0673. Min Act: -0.1700

Data Index: 280032 (C4 (Web Text))

Max Activating Token Index: 855

Text #1

Max Range: 4.0673. Min Range: -4.0673

Max Act: 4.0360. Min Act: -0.1700

Data Index: 1108348 (C4 (Web Text))

Max Activating Token Index: 921

Text #2

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.9110. Min Act: -0.1700

Data Index: 1119952 (C4 (Web Text))

Max Activating Token Index: 990

Text #3

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.9110. Min Act: -0.1700

Data Index: 326456 (C4 (Web Text))

Max Activating Token Index: 616

Text #4

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.8797. Min Act: -0.1700

Data Index: 1051590 (C4 (Web Text))

Max Activating Token Index: 335

Text #5

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.9110. Min Act: -0.1700

Data Index: 452830 (C4 (Web Text))

Max Activating Token Index: 76

Text #6

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.8797. Min Act: -0.1700

Data Index: 744736 (C4 (Web Text))

Max Activating Token Index: 193

Text #7

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.8484. Min Act: -0.1700

Data Index: 1337108 (C4 (Web Text))

Max Activating Token Index: 211

Text #8

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.8171. Min Act: -0.1700

Data Index: 138694 (C4 (Web Text))

Max Activating Token Index: 100

Text #9

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.8171. Min Act: -0.1700

Data Index: 27559 (C4 (Web Text))

Max Activating Token Index: 878

Text #10

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.8484. Min Act: -0.1700

Data Index: 77802 (C4 (Web Text))

Max Activating Token Index: 790

Text #11

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.7859. Min Act: -0.1700

Data Index: 1127656 (C4 (Web Text))

Max Activating Token Index: 880

Text #12

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.8171. Min Act: -0.1700

Data Index: 484744 (C4 (Web Text))

Max Activating Token Index: 857

Text #13

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.7546. Min Act: -0.1700

Data Index: 714004 (C4 (Web Text))

Max Activating Token Index: 562

Text #14

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.7859. Min Act: -0.1700

Data Index: 602455 (C4 (Web Text))

Max Activating Token Index: 672

Text #15

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.7859. Min Act: -0.1700

Data Index: 408995 (C4 (Web Text))

Max Activating Token Index: 557

Text #16

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.7546. Min Act: -0.1700

Data Index: 760287 (C4 (Web Text))

Max Activating Token Index: 286

Text #17

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.7546. Min Act: -0.1700

Data Index: 727514 (C4 (Web Text))

Max Activating Token Index: 25

Text #18

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.6920. Min Act: -0.1700

Data Index: 750201 (C4 (Web Text))

Max Activating Token Index: 626

Text #19

Max Range: 4.0673. Min Range: -4.0673

Max Act: 3.7233. Min Act: -0.1700

Data Index: 621282 (C4 (Web Text))

Max Activating Token Index: 491