Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.1784. Min Act: -0.1700

Data Index: 151888 (C4 (Web Text))

Max Activating Token Index: 251

Text #1

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.1478. Min Act: -0.1700

Data Index: 679687 (C4 (Web Text))

Max Activating Token Index: 73

Text #2

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.1594. Min Act: -0.1700

Data Index: 765205 (C4 (Web Text))

Max Activating Token Index: 27

Text #3

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.1370. Min Act: -0.1700

Data Index: 542652 (C4 (Web Text))

Max Activating Token Index: 206

Text #4

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.1141. Min Act: -0.1700

Data Index: 911419 (C4 (Web Text))

Max Activating Token Index: 528

Text #5

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.1135. Min Act: -0.1700

Data Index: 460974 (C4 (Web Text))

Max Activating Token Index: 760

Text #6

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.1020. Min Act: -0.1700

Data Index: 216724 (C4 (Web Text))

Max Activating Token Index: 1017

Text #7

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0920. Min Act: -0.1700

Data Index: 1066981 (C4 (Web Text))

Max Activating Token Index: 390

Text #8

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0835. Min Act: -0.1700

Data Index: 21902 (C4 (Web Text))

Max Activating Token Index: 710

Text #9

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0709. Min Act: -0.1700

Data Index: 1562589 (Python Code)

Max Activating Token Index: 381

Text #10

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0900. Min Act: -0.1700

Data Index: 1324899 (C4 (Web Text))

Max Activating Token Index: 1012

Text #11

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0806. Min Act: -0.1700

Data Index: 232947 (C4 (Web Text))

Max Activating Token Index: 749

Text #12

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0323. Min Act: -0.1700

Data Index: 1226306 (C4 (Web Text))

Max Activating Token Index: 683

Text #13

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0565. Min Act: -0.1700

Data Index: 651326 (C4 (Web Text))

Max Activating Token Index: 12

Text #14

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0358. Min Act: -0.1700

Data Index: 587485 (C4 (Web Text))

Max Activating Token Index: 804

Text #15

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0134. Min Act: -0.1700

Data Index: 364766 (C4 (Web Text))

Max Activating Token Index: 548

Text #16

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0094. Min Act: -0.1700

Data Index: 1322659 (C4 (Web Text))

Max Activating Token Index: 812

Text #17

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0018. Min Act: -0.1700

Data Index: 60748 (C4 (Web Text))

Max Activating Token Index: 363

Text #18

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0298. Min Act: -0.1700

Data Index: 1086212 (C4 (Web Text))

Max Activating Token Index: 1001

Text #19

Max Range: 3.1784. Min Range: -3.1784

Max Act: 3.0311. Min Act: -0.1700

Data Index: 600603 (C4 (Web Text))

Max Activating Token Index: 153