Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1970 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.1157. Min Range: -3.1157

Max Act: 3.1157. Min Act: -0.1700

Data Index: 543712 (C4 (Web Text))

Max Activating Token Index: 144

Text #1

Max Range: 3.1157. Min Range: -3.1157

Max Act: 3.0574. Min Act: -0.1700

Data Index: 119570 (C4 (Web Text))

Max Activating Token Index: 842

Text #2

Max Range: 3.1157. Min Range: -3.1157

Max Act: 3.0445. Min Act: -0.1700

Data Index: 141865 (C4 (Web Text))

Max Activating Token Index: 241

Text #3

Max Range: 3.1157. Min Range: -3.1157

Max Act: 3.0015. Min Act: -0.1699

Data Index: 1458738 (Python Code)

Max Activating Token Index: 254

Text #4

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.9879. Min Act: -0.1700

Data Index: 523304 (C4 (Web Text))

Max Activating Token Index: 710

Text #5

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.9582. Min Act: -0.1700

Data Index: 543240 (C4 (Web Text))

Max Activating Token Index: 18

Text #6

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.9488. Min Act: -0.1700

Data Index: 651593 (C4 (Web Text))

Max Activating Token Index: 412

Text #7

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.9371. Min Act: -0.1700

Data Index: 964479 (C4 (Web Text))

Max Activating Token Index: 833

Text #8

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.9241. Min Act: -0.1700

Data Index: 194877 (C4 (Web Text))

Max Activating Token Index: 127

Text #9

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.9147. Min Act: -0.1700

Data Index: 1123694 (C4 (Web Text))

Max Activating Token Index: 300

Text #10

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.9216. Min Act: -0.1700

Data Index: 310764 (C4 (Web Text))

Max Activating Token Index: 905

Text #11

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8911. Min Act: -0.1699

Data Index: 1359998 (Python Code)

Max Activating Token Index: 251

Text #12

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8925. Min Act: -0.1700

Data Index: 777791 (C4 (Web Text))

Max Activating Token Index: 224

Text #13

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8750. Min Act: -0.1700

Data Index: 18629 (C4 (Web Text))

Max Activating Token Index: 786

Text #14

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8955. Min Act: -0.1700

Data Index: 536246 (C4 (Web Text))

Max Activating Token Index: 132

Text #15

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8759. Min Act: -0.1700

Data Index: 1163592 (C4 (Web Text))

Max Activating Token Index: 28

Text #16

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8573. Min Act: -0.1700

Data Index: 728109 (C4 (Web Text))

Max Activating Token Index: 656

Text #17

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8699. Min Act: -0.1700

Data Index: 679494 (C4 (Web Text))

Max Activating Token Index: 675

Text #18

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8626. Min Act: -0.1700

Data Index: 103180 (C4 (Web Text))

Max Activating Token Index: 1011

Text #19

Max Range: 3.1157. Min Range: -3.1157

Max Act: 2.8639. Min Act: -0.1700

Data Index: 618922 (C4 (Web Text))

Max Activating Token Index: 107