Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1417 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.3552. Min Act: -0.1700

Data Index: 1066442 (C4 (Web Text))

Max Activating Token Index: 727

Text #1

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.3178. Min Act: -0.1700

Data Index: 163055 (C4 (Web Text))

Max Activating Token Index: 393

Text #2

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2568. Min Act: -0.1700

Data Index: 1119344 (C4 (Web Text))

Max Activating Token Index: 967

Text #3

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1975. Min Act: -0.1700

Data Index: 1649379 (Python Code)

Max Activating Token Index: 404

Text #4

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2078. Min Act: -0.1700

Data Index: 1339412 (C4 (Web Text))

Max Activating Token Index: 168

Text #5

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2037. Min Act: -0.1700

Data Index: 834979 (C4 (Web Text))

Max Activating Token Index: 537

Text #6

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1894. Min Act: -0.1700

Data Index: 715778 (C4 (Web Text))

Max Activating Token Index: 77

Text #7

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1818. Min Act: -0.1700

Data Index: 1496550 (Python Code)

Max Activating Token Index: 801

Text #8

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1773. Min Act: -0.1700

Data Index: 1318896 (C4 (Web Text))

Max Activating Token Index: 363

Text #9

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1280. Min Act: -0.1700

Data Index: 1541090 (Python Code)

Max Activating Token Index: 351

Text #10

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1315. Min Act: -0.1700

Data Index: 1647640 (Python Code)

Max Activating Token Index: 72

Text #11

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1014. Min Act: -0.1700

Data Index: 927772 (C4 (Web Text))

Max Activating Token Index: 915

Text #12

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1206. Min Act: -0.1700

Data Index: 193470 (C4 (Web Text))

Max Activating Token Index: 238

Text #13

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1204. Min Act: -0.1700

Data Index: 152748 (C4 (Web Text))

Max Activating Token Index: 384

Text #14

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1178. Min Act: -0.1700

Data Index: 1482634 (Python Code)

Max Activating Token Index: 902

Text #15

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1150. Min Act: -0.1700

Data Index: 534924 (C4 (Web Text))

Max Activating Token Index: 521

Text #16

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.0900. Min Act: -0.1700

Data Index: 282977 (C4 (Web Text))

Max Activating Token Index: 53

Text #17

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.1119. Min Act: -0.1700

Data Index: 653364 (C4 (Web Text))

Max Activating Token Index: 795

Text #18

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.0848. Min Act: -0.1700

Data Index: 1593001 (Python Code)

Max Activating Token Index: 647

Text #19

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.0680. Min Act: -0.1700

Data Index: 1373870 (Python Code)

Max Activating Token Index: 76