Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 777 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2675. Min Act: -0.1700

Data Index: 453041 (C4 (Web Text))

Max Activating Token Index: 968

Text #1

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2810. Min Act: -0.1699

Data Index: 658596 (C4 (Web Text))

Max Activating Token Index: 101

Text #2

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2894. Min Act: -0.1700

Data Index: 1200915 (C4 (Web Text))

Max Activating Token Index: 968

Text #3

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2697. Min Act: -0.1700

Data Index: 147402 (C4 (Web Text))

Max Activating Token Index: 36

Text #4

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2671. Min Act: -0.1700

Data Index: 466338 (C4 (Web Text))

Max Activating Token Index: 118

Text #5

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2762. Min Act: -0.1700

Data Index: 1114665 (C4 (Web Text))

Max Activating Token Index: 605

Text #6

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2644. Min Act: -0.1700

Data Index: 1315781 (C4 (Web Text))

Max Activating Token Index: 926

Text #7

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2559. Min Act: -0.1700

Data Index: 1336081 (C4 (Web Text))

Max Activating Token Index: 762

Text #8

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2529. Min Act: -0.1700

Data Index: 19217 (C4 (Web Text))

Max Activating Token Index: 720

Text #9

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2400. Min Act: -0.1700

Data Index: 1302747 (C4 (Web Text))

Max Activating Token Index: 722

Text #10

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2413. Min Act: -0.1700

Data Index: 486882 (C4 (Web Text))

Max Activating Token Index: 546

Text #11

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2444. Min Act: -0.1700

Data Index: 515378 (C4 (Web Text))

Max Activating Token Index: 11

Text #12

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2543. Min Act: -0.1700

Data Index: 673343 (C4 (Web Text))

Max Activating Token Index: 665

Text #13

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2327. Min Act: -0.1700

Data Index: 1129889 (C4 (Web Text))

Max Activating Token Index: 349

Text #14

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2501. Min Act: -0.1700

Data Index: 629036 (C4 (Web Text))

Max Activating Token Index: 752

Text #15

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2342. Min Act: -0.1700

Data Index: 1323518 (C4 (Web Text))

Max Activating Token Index: 677

Text #16

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2223. Min Act: -0.1700

Data Index: 1151730 (C4 (Web Text))

Max Activating Token Index: 504

Text #17

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2195. Min Act: -0.1700

Data Index: 537265 (C4 (Web Text))

Max Activating Token Index: 860

Text #18

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2200. Min Act: -0.1700

Data Index: 837079 (C4 (Web Text))

Max Activating Token Index: 561

Text #19

Max Range: 3.2894. Min Range: -3.2894

Max Act: 3.2227. Min Act: -0.1700

Data Index: 1086427 (C4 (Web Text))

Max Activating Token Index: 610