Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1087 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.4906. Min Act: -0.1700

Data Index: 1303973 (C4 (Web Text))

Max Activating Token Index: 118

Text #1

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.4788. Min Act: -0.1700

Data Index: 1276170 (C4 (Web Text))

Max Activating Token Index: 9

Text #2

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.4353. Min Act: -0.1700

Data Index: 525985 (C4 (Web Text))

Max Activating Token Index: 712

Text #3

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.4239. Min Act: -0.1700

Data Index: 398767 (C4 (Web Text))

Max Activating Token Index: 146

Text #4

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.4160. Min Act: -0.1700

Data Index: 1275325 (C4 (Web Text))

Max Activating Token Index: 292

Text #5

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3997. Min Act: -0.1700

Data Index: 744053 (C4 (Web Text))

Max Activating Token Index: 504

Text #6

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3864. Min Act: -0.1700

Data Index: 385841 (C4 (Web Text))

Max Activating Token Index: 232

Text #7

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3616. Min Act: -0.1700

Data Index: 500706 (C4 (Web Text))

Max Activating Token Index: 839

Text #8

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3523. Min Act: -0.1700

Data Index: 250426 (C4 (Web Text))

Max Activating Token Index: 550

Text #9

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3311. Min Act: -0.1700

Data Index: 917863 (C4 (Web Text))

Max Activating Token Index: 885

Text #10

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3381. Min Act: -0.1700

Data Index: 716063 (C4 (Web Text))

Max Activating Token Index: 654

Text #11

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3169. Min Act: -0.1700

Data Index: 96479 (C4 (Web Text))

Max Activating Token Index: 453

Text #12

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3046. Min Act: -0.1700

Data Index: 1189273 (C4 (Web Text))

Max Activating Token Index: 247

Text #13

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.2981. Min Act: -0.1700

Data Index: 668774 (C4 (Web Text))

Max Activating Token Index: 948

Text #14

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.3017. Min Act: -0.1700

Data Index: 561446 (C4 (Web Text))

Max Activating Token Index: 359

Text #15

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.2734. Min Act: -0.1700

Data Index: 724959 (C4 (Web Text))

Max Activating Token Index: 215

Text #16

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.2836. Min Act: -0.1700

Data Index: 1075854 (C4 (Web Text))

Max Activating Token Index: 254

Text #17

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.2937. Min Act: -0.1700

Data Index: 1060128 (C4 (Web Text))

Max Activating Token Index: 979

Text #18

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.2785. Min Act: -0.1700

Data Index: 1051341 (C4 (Web Text))

Max Activating Token Index: 436

Text #19

Max Range: 3.4906. Min Range: -3.4906

Max Act: 3.2763. Min Act: -0.1700

Data Index: 454626 (C4 (Web Text))

Max Activating Token Index: 257