Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1412 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 5.0363. Min Range: -5.0363

Max Act: 5.0363. Min Act: -0.1700

Data Index: 128304 (C4 (Web Text))

Max Activating Token Index: 453

Text #1

Max Range: 5.0363. Min Range: -5.0363

Max Act: 5.0066. Min Act: -0.1700

Data Index: 1111776 (C4 (Web Text))

Max Activating Token Index: 985

Text #2

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.9785. Min Act: -0.1700

Data Index: 713844 (C4 (Web Text))

Max Activating Token Index: 867

Text #3

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.9351. Min Act: -0.1700

Data Index: 1304151 (C4 (Web Text))

Max Activating Token Index: 135

Text #4

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.9047. Min Act: -0.1700

Data Index: 57903 (C4 (Web Text))

Max Activating Token Index: 400

Text #5

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8967. Min Act: -0.1700

Data Index: 176359 (C4 (Web Text))

Max Activating Token Index: 846

Text #6

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8823. Min Act: -0.1700

Data Index: 902624 (C4 (Web Text))

Max Activating Token Index: 598

Text #7

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8633. Min Act: -0.1700

Data Index: 1188660 (C4 (Web Text))

Max Activating Token Index: 734

Text #8

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8484. Min Act: -0.1700

Data Index: 1260660 (C4 (Web Text))

Max Activating Token Index: 296

Text #9

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8507. Min Act: -0.1700

Data Index: 45807 (C4 (Web Text))

Max Activating Token Index: 730

Text #10

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8251. Min Act: -0.1700

Data Index: 387091 (C4 (Web Text))

Max Activating Token Index: 881

Text #11

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8223. Min Act: -0.1700

Data Index: 446312 (C4 (Web Text))

Max Activating Token Index: 63

Text #12

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8238. Min Act: -0.1700

Data Index: 963318 (C4 (Web Text))

Max Activating Token Index: 640

Text #13

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8245. Min Act: -0.1700

Data Index: 1061568 (C4 (Web Text))

Max Activating Token Index: 88

Text #14

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.7874. Min Act: -0.1700

Data Index: 1048298 (C4 (Web Text))

Max Activating Token Index: 900

Text #15

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.7997. Min Act: -0.1700

Data Index: 418372 (C4 (Web Text))

Max Activating Token Index: 691

Text #16

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.7996. Min Act: -0.1700

Data Index: 112467 (C4 (Web Text))

Max Activating Token Index: 583

Text #17

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8095. Min Act: -0.1700

Data Index: 1138999 (C4 (Web Text))

Max Activating Token Index: 478

Text #18

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.7780. Min Act: -0.1700

Data Index: 906380 (C4 (Web Text))

Max Activating Token Index: 610

Text #19

Max Range: 5.0363. Min Range: -5.0363

Max Act: 4.8275. Min Act: -0.1700

Data Index: 22670 (C4 (Web Text))

Max Activating Token Index: 976