Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1288 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.6202. Min Act: -0.1700

Data Index: 398964 (C4 (Web Text))

Max Activating Token Index: 494

Text #1

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.5561. Min Act: -0.1700

Data Index: 637093 (C4 (Web Text))

Max Activating Token Index: 358

Text #2

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.5347. Min Act: -0.1700

Data Index: 1303401 (C4 (Web Text))

Max Activating Token Index: 713

Text #3

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.4551. Min Act: -0.1700

Data Index: 1257691 (C4 (Web Text))

Max Activating Token Index: 130

Text #4

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.4689. Min Act: -0.1700

Data Index: 351024 (C4 (Web Text))

Max Activating Token Index: 556

Text #5

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.4476. Min Act: -0.1700

Data Index: 387471 (C4 (Web Text))

Max Activating Token Index: 513

Text #6

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3940. Min Act: -0.1700

Data Index: 432304 (C4 (Web Text))

Max Activating Token Index: 9

Text #7

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3965. Min Act: -0.1700

Data Index: 92602 (C4 (Web Text))

Max Activating Token Index: 969

Text #8

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3828. Min Act: -0.1700

Data Index: 323470 (C4 (Web Text))

Max Activating Token Index: 435

Text #9

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3810. Min Act: -0.1700

Data Index: 45052 (C4 (Web Text))

Max Activating Token Index: 386

Text #10

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3468. Min Act: -0.1700

Data Index: 153144 (C4 (Web Text))

Max Activating Token Index: 660

Text #11

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3347. Min Act: -0.1700

Data Index: 997662 (C4 (Web Text))

Max Activating Token Index: 22

Text #12

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3213. Min Act: -0.1700

Data Index: 158285 (C4 (Web Text))

Max Activating Token Index: 274

Text #13

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3069. Min Act: -0.1700

Data Index: 202384 (C4 (Web Text))

Max Activating Token Index: 1004

Text #14

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.3020. Min Act: -0.1700

Data Index: 657549 (C4 (Web Text))

Max Activating Token Index: 495

Text #15

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.2842. Min Act: -0.1700

Data Index: 1319836 (C4 (Web Text))

Max Activating Token Index: 979

Text #16

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.2838. Min Act: -0.1700

Data Index: 230692 (C4 (Web Text))

Max Activating Token Index: 423

Text #17

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.2799. Min Act: -0.1700

Data Index: 795264 (C4 (Web Text))

Max Activating Token Index: 790

Text #18

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.2796. Min Act: -0.1700

Data Index: 38250 (C4 (Web Text))

Max Activating Token Index: 148

Text #19

Max Range: 3.6202. Min Range: -3.6202

Max Act: 3.2672. Min Act: -0.1700

Data Index: 1206281 (C4 (Web Text))

Max Activating Token Index: 947