Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 59 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.0507. Min Range: -3.0507

Max Act: 3.0507. Min Act: -0.1700

Data Index: 81762 (C4 (Web Text))

Max Activating Token Index: 371

Text #1

Max Range: 3.0507. Min Range: -3.0507

Max Act: 3.0478. Min Act: -0.1700

Data Index: 288627 (C4 (Web Text))

Max Activating Token Index: 528

Text #2

Max Range: 3.0507. Min Range: -3.0507

Max Act: 3.0379. Min Act: -0.1700

Data Index: 654981 (C4 (Web Text))

Max Activating Token Index: 588

Text #3

Max Range: 3.0507. Min Range: -3.0507

Max Act: 3.0078. Min Act: -0.1700

Data Index: 810633 (C4 (Web Text))

Max Activating Token Index: 279

Text #4

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9901. Min Act: -0.1700

Data Index: 925784 (C4 (Web Text))

Max Activating Token Index: 695

Text #5

Max Range: 3.0507. Min Range: -3.0507

Max Act: 3.0018. Min Act: -0.1700

Data Index: 384459 (C4 (Web Text))

Max Activating Token Index: 904

Text #6

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9881. Min Act: -0.1700

Data Index: 478278 (C4 (Web Text))

Max Activating Token Index: 374

Text #7

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9800. Min Act: -0.1700

Data Index: 135827 (C4 (Web Text))

Max Activating Token Index: 679

Text #8

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9704. Min Act: -0.1700

Data Index: 868533 (C4 (Web Text))

Max Activating Token Index: 747

Text #9

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9178. Min Act: -0.1700

Data Index: 378068 (C4 (Web Text))

Max Activating Token Index: 119

Text #10

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9225. Min Act: -0.1700

Data Index: 66418 (C4 (Web Text))

Max Activating Token Index: 226

Text #11

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9124. Min Act: -0.1700

Data Index: 109090 (C4 (Web Text))

Max Activating Token Index: 960

Text #12

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9243. Min Act: -0.1700

Data Index: 1347635 (C4 (Web Text))

Max Activating Token Index: 691

Text #13

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9127. Min Act: -0.1700

Data Index: 885027 (C4 (Web Text))

Max Activating Token Index: 534

Text #14

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9132. Min Act: -0.1700

Data Index: 748052 (C4 (Web Text))

Max Activating Token Index: 462

Text #15

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9069. Min Act: -0.1700

Data Index: 261388 (C4 (Web Text))

Max Activating Token Index: 235

Text #16

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9236. Min Act: -0.1700

Data Index: 1163135 (C4 (Web Text))

Max Activating Token Index: 412

Text #17

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9039. Min Act: -0.1700

Data Index: 447684 (C4 (Web Text))

Max Activating Token Index: 60

Text #18

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.8734. Min Act: -0.1700

Data Index: 278876 (C4 (Web Text))

Max Activating Token Index: 924

Text #19

Max Range: 3.0507. Min Range: -3.0507

Max Act: 2.9036. Min Act: -0.1700

Data Index: 998521 (C4 (Web Text))

Max Activating Token Index: 78