Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1171 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.1470. Min Range: -4.1470

Max Act: 4.1470. Min Act: -0.1700

Data Index: 1032961 (C4 (Web Text))

Max Activating Token Index: 307

Text #1

Max Range: 4.1470. Min Range: -4.1470

Max Act: 4.1110. Min Act: -0.1695

Data Index: 1459146 (Python Code)

Max Activating Token Index: 529

Text #2

Max Range: 4.1470. Min Range: -4.1470

Max Act: 4.1062. Min Act: -0.1700

Data Index: 549956 (C4 (Web Text))

Max Activating Token Index: 527

Text #3

Max Range: 4.1470. Min Range: -4.1470

Max Act: 4.1105. Min Act: -0.1700

Data Index: 1610567 (Python Code)

Max Activating Token Index: 927

Text #4

Max Range: 4.1470. Min Range: -4.1470

Max Act: 4.0526. Min Act: -0.1700

Data Index: 949942 (C4 (Web Text))

Max Activating Token Index: 208

Text #5

Max Range: 4.1470. Min Range: -4.1470

Max Act: 4.0302. Min Act: -0.1699

Data Index: 649656 (C4 (Web Text))

Max Activating Token Index: 159

Text #6

Max Range: 4.1470. Min Range: -4.1470

Max Act: 4.0313. Min Act: -0.1692

Data Index: 932560 (C4 (Web Text))

Max Activating Token Index: 856

Text #7

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.9990. Min Act: -0.1698

Data Index: 142022 (C4 (Web Text))

Max Activating Token Index: 807

Text #8

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.9934. Min Act: -0.1697

Data Index: 532401 (C4 (Web Text))

Max Activating Token Index: 387

Text #9

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.9481. Min Act: -0.1698

Data Index: 1336721 (C4 (Web Text))

Max Activating Token Index: 611

Text #10

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.9488. Min Act: -0.1700

Data Index: 841731 (C4 (Web Text))

Max Activating Token Index: 417

Text #11

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8905. Min Act: -0.1699

Data Index: 4414 (C4 (Web Text))

Max Activating Token Index: 389

Text #12

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.9022. Min Act: -0.1700

Data Index: 108099 (C4 (Web Text))

Max Activating Token Index: 583

Text #13

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8819. Min Act: -0.1677

Data Index: 97407 (C4 (Web Text))

Max Activating Token Index: 405

Text #14

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8695. Min Act: -0.1699

Data Index: 603644 (C4 (Web Text))

Max Activating Token Index: 401

Text #15

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8549. Min Act: -0.1700

Data Index: 680352 (C4 (Web Text))

Max Activating Token Index: 104

Text #16

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8499. Min Act: -0.1700

Data Index: 1555704 (Python Code)

Max Activating Token Index: 537

Text #17

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8382. Min Act: -0.1699

Data Index: 213366 (C4 (Web Text))

Max Activating Token Index: 471

Text #18

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8240. Min Act: -0.1699

Data Index: 241681 (C4 (Web Text))

Max Activating Token Index: 65

Text #19

Max Range: 4.1470. Min Range: -4.1470

Max Act: 3.8074. Min Act: -0.1700

Data Index: 381686 (C4 (Web Text))

Max Activating Token Index: 45