Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1096 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.6047. Min Act: -0.1699

Data Index: 313422 (C4 (Web Text))

Max Activating Token Index: 21

Text #1

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.5841. Min Act: -0.1700

Data Index: 346884 (C4 (Web Text))

Max Activating Token Index: 69

Text #2

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.5444. Min Act: -0.1698

Data Index: 744836 (C4 (Web Text))

Max Activating Token Index: 264

Text #3

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.4956. Min Act: -0.1700

Data Index: 979129 (C4 (Web Text))

Max Activating Token Index: 665

Text #4

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.4563. Min Act: -0.1700

Data Index: 453294 (C4 (Web Text))

Max Activating Token Index: 10

Text #5

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.4292. Min Act: -0.1700

Data Index: 452855 (C4 (Web Text))

Max Activating Token Index: 619

Text #6

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.4442. Min Act: -0.1700

Data Index: 417147 (C4 (Web Text))

Max Activating Token Index: 222

Text #7

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.4181. Min Act: -0.1695

Data Index: 365902 (C4 (Web Text))

Max Activating Token Index: 822

Text #8

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.4038. Min Act: -0.1688

Data Index: 61931 (C4 (Web Text))

Max Activating Token Index: 304

Text #9

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3941. Min Act: -0.1700

Data Index: 1127291 (C4 (Web Text))

Max Activating Token Index: 436

Text #10

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3927. Min Act: -0.1700

Data Index: 1054008 (C4 (Web Text))

Max Activating Token Index: 791

Text #11

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.4081. Min Act: -0.1695

Data Index: 902269 (C4 (Web Text))

Max Activating Token Index: 78

Text #12

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3703. Min Act: -0.1700

Data Index: 1098735 (C4 (Web Text))

Max Activating Token Index: 506

Text #13

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3707. Min Act: -0.1698

Data Index: 1144057 (C4 (Web Text))

Max Activating Token Index: 642

Text #14

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3386. Min Act: -0.1700

Data Index: 1287451 (C4 (Web Text))

Max Activating Token Index: 857

Text #15

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3477. Min Act: -0.1700

Data Index: 388420 (C4 (Web Text))

Max Activating Token Index: 28

Text #16

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3474. Min Act: -0.1700

Data Index: 90214 (C4 (Web Text))

Max Activating Token Index: 660

Text #17

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3632. Min Act: -0.1700

Data Index: 837032 (C4 (Web Text))

Max Activating Token Index: 982

Text #18

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3594. Min Act: -0.1700

Data Index: 434892 (C4 (Web Text))

Max Activating Token Index: 78

Text #19

Max Range: 3.6047. Min Range: -3.6047

Max Act: 3.3495. Min Act: -0.1686

Data Index: 1013417 (C4 (Web Text))

Max Activating Token Index: 391