Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 390 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.4048. Min Act: -0.1700

Data Index: 102829 (C4 (Web Text))

Max Activating Token Index: 915

Text #1

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.4022. Min Act: -0.1700

Data Index: 136035 (C4 (Web Text))

Max Activating Token Index: 370

Text #2

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3555. Min Act: -0.1700

Data Index: 328058 (C4 (Web Text))

Max Activating Token Index: 456

Text #3

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3741. Min Act: -0.1700

Data Index: 487366 (C4 (Web Text))

Max Activating Token Index: 866

Text #4

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3441. Min Act: -0.1700

Data Index: 823252 (C4 (Web Text))

Max Activating Token Index: 182

Text #5

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3493. Min Act: -0.1700

Data Index: 1072314 (C4 (Web Text))

Max Activating Token Index: 797

Text #6

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3335. Min Act: -0.1700

Data Index: 12096 (C4 (Web Text))

Max Activating Token Index: 799

Text #7

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3173. Min Act: -0.1700

Data Index: 101873 (C4 (Web Text))

Max Activating Token Index: 516

Text #8

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3154. Min Act: -0.1700

Data Index: 1128616 (C4 (Web Text))

Max Activating Token Index: 407

Text #9

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3093. Min Act: -0.1700

Data Index: 53144 (C4 (Web Text))

Max Activating Token Index: 710

Text #10

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3080. Min Act: -0.1700

Data Index: 1193460 (C4 (Web Text))

Max Activating Token Index: 436

Text #11

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2995. Min Act: -0.1700

Data Index: 671161 (C4 (Web Text))

Max Activating Token Index: 420

Text #12

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.3018. Min Act: -0.1700

Data Index: 16197 (C4 (Web Text))

Max Activating Token Index: 911

Text #13

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2930. Min Act: -0.1700

Data Index: 602259 (C4 (Web Text))

Max Activating Token Index: 548

Text #14

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2815. Min Act: -0.1700

Data Index: 567957 (C4 (Web Text))

Max Activating Token Index: 752

Text #15

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2806. Min Act: -0.1700

Data Index: 907146 (C4 (Web Text))

Max Activating Token Index: 458

Text #16

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2720. Min Act: -0.1700

Data Index: 515568 (C4 (Web Text))

Max Activating Token Index: 518

Text #17

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2664. Min Act: -0.1700

Data Index: 126269 (C4 (Web Text))

Max Activating Token Index: 473

Text #18

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2815. Min Act: -0.1700

Data Index: 381622 (C4 (Web Text))

Max Activating Token Index: 122

Text #19

Max Range: 3.4048. Min Range: -3.4048

Max Act: 3.2519. Min Act: -0.1700

Data Index: 788821 (C4 (Web Text))

Max Activating Token Index: 324