Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1158 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.5746. Min Act: -0.1700

Data Index: 1005782 (C4 (Web Text))

Max Activating Token Index: 151

Text #1

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.4514. Min Act: -0.1700

Data Index: 351457 (C4 (Web Text))

Max Activating Token Index: 15

Text #2

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.4131. Min Act: -0.1700

Data Index: 121916 (C4 (Web Text))

Max Activating Token Index: 689

Text #3

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3968. Min Act: -0.1700

Data Index: 60318 (C4 (Web Text))

Max Activating Token Index: 833

Text #4

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3710. Min Act: -0.1700

Data Index: 725115 (C4 (Web Text))

Max Activating Token Index: 887

Text #5

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3563. Min Act: -0.1700

Data Index: 723242 (C4 (Web Text))

Max Activating Token Index: 131

Text #6

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3464. Min Act: -0.1700

Data Index: 1318639 (C4 (Web Text))

Max Activating Token Index: 22

Text #7

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3435. Min Act: -0.1700

Data Index: 1110437 (C4 (Web Text))

Max Activating Token Index: 789

Text #8

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3476. Min Act: -0.1700

Data Index: 771894 (C4 (Web Text))

Max Activating Token Index: 684

Text #9

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3466. Min Act: -0.1700

Data Index: 1274135 (C4 (Web Text))

Max Activating Token Index: 157

Text #10

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3348. Min Act: -0.1700

Data Index: 923067 (C4 (Web Text))

Max Activating Token Index: 319

Text #11

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3155. Min Act: -0.1700

Data Index: 317214 (C4 (Web Text))

Max Activating Token Index: 464

Text #12

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3131. Min Act: -0.1700

Data Index: 926584 (C4 (Web Text))

Max Activating Token Index: 866

Text #13

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3229. Min Act: -0.1700

Data Index: 613828 (C4 (Web Text))

Max Activating Token Index: 674

Text #14

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3195. Min Act: -0.1700

Data Index: 1049328 (C4 (Web Text))

Max Activating Token Index: 700

Text #15

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3223. Min Act: -0.1700

Data Index: 650033 (C4 (Web Text))

Max Activating Token Index: 342

Text #16

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.2930. Min Act: -0.1700

Data Index: 749219 (C4 (Web Text))

Max Activating Token Index: 121

Text #17

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.2932. Min Act: -0.1700

Data Index: 764336 (C4 (Web Text))

Max Activating Token Index: 722

Text #18

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.3029. Min Act: -0.1700

Data Index: 747680 (C4 (Web Text))

Max Activating Token Index: 910

Text #19

Max Range: 3.5746. Min Range: -3.5746

Max Act: 3.2886. Min Act: -0.1700

Data Index: 345043 (C4 (Web Text))

Max Activating Token Index: 256