Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1119 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.8138. Min Act: -0.1700

Data Index: 771353 (C4 (Web Text))

Max Activating Token Index: 89

Text #1

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.7191. Min Act: -0.1694

Data Index: 1102999 (C4 (Web Text))

Max Activating Token Index: 622

Text #2

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.7111. Min Act: -0.1700

Data Index: 1224426 (C4 (Web Text))

Max Activating Token Index: 225

Text #3

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6596. Min Act: -0.1655

Data Index: 564622 (C4 (Web Text))

Max Activating Token Index: 748

Text #4

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6418. Min Act: -0.1679

Data Index: 841966 (C4 (Web Text))

Max Activating Token Index: 535

Text #5

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6505. Min Act: -0.1700

Data Index: 963786 (C4 (Web Text))

Max Activating Token Index: 31

Text #6

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6419. Min Act: -0.1700

Data Index: 728223 (C4 (Web Text))

Max Activating Token Index: 33

Text #7

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6417. Min Act: -0.1700

Data Index: 1259830 (C4 (Web Text))

Max Activating Token Index: 961

Text #8

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6218. Min Act: -0.1700

Data Index: 969783 (C4 (Web Text))

Max Activating Token Index: 952

Text #9

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6234. Min Act: -0.1699

Data Index: 1363808 (Python Code)

Max Activating Token Index: 762

Text #10

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6054. Min Act: -0.1700

Data Index: 653059 (C4 (Web Text))

Max Activating Token Index: 750

Text #11

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.6190. Min Act: -0.1700

Data Index: 1262001 (C4 (Web Text))

Max Activating Token Index: 77

Text #12

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5897. Min Act: -0.1700

Data Index: 726789 (C4 (Web Text))

Max Activating Token Index: 552

Text #13

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5974. Min Act: -0.1696

Data Index: 391818 (C4 (Web Text))

Max Activating Token Index: 32

Text #14

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5978. Min Act: -0.1700

Data Index: 474296 (C4 (Web Text))

Max Activating Token Index: 793

Text #15

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5812. Min Act: -0.1700

Data Index: 70606 (C4 (Web Text))

Max Activating Token Index: 584

Text #16

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5889. Min Act: -0.1700

Data Index: 639922 (C4 (Web Text))

Max Activating Token Index: 398

Text #17

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5651. Min Act: -0.1700

Data Index: 907130 (C4 (Web Text))

Max Activating Token Index: 879

Text #18

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5635. Min Act: -0.1700

Data Index: 1242193 (C4 (Web Text))

Max Activating Token Index: 601

Text #19

Max Range: 2.8138. Min Range: -2.8138

Max Act: 2.5641. Min Act: -0.1687

Data Index: 135738 (C4 (Web Text))

Max Activating Token Index: 774