Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 752 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.1794. Min Act: -0.1700

Data Index: 1242295 (C4 (Web Text))

Max Activating Token Index: 840

Text #1

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.1733. Min Act: -0.1700

Data Index: 1003207 (C4 (Web Text))

Max Activating Token Index: 765

Text #2

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.1809. Min Act: -0.1700

Data Index: 1041767 (C4 (Web Text))

Max Activating Token Index: 639

Text #3

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.1615. Min Act: -0.1700

Data Index: 520313 (C4 (Web Text))

Max Activating Token Index: 915

Text #4

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.1469. Min Act: -0.1700

Data Index: 73492 (C4 (Web Text))

Max Activating Token Index: 224

Text #5

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.1524. Min Act: -0.1700

Data Index: 956221 (C4 (Web Text))

Max Activating Token Index: 520

Text #6

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.1498. Min Act: -0.1700

Data Index: 688774 (C4 (Web Text))

Max Activating Token Index: 964

Text #7

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0630. Min Act: -0.1700

Data Index: 950595 (C4 (Web Text))

Max Activating Token Index: 789

Text #8

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0745. Min Act: -0.1700

Data Index: 1358734 (C4 (Web Text))

Max Activating Token Index: 447

Text #9

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0687. Min Act: -0.1700

Data Index: 178951 (C4 (Web Text))

Max Activating Token Index: 171

Text #10

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0348. Min Act: -0.1700

Data Index: 710073 (C4 (Web Text))

Max Activating Token Index: 102

Text #11

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0104. Min Act: -0.1700

Data Index: 36056 (C4 (Web Text))

Max Activating Token Index: 461

Text #12

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0288. Min Act: -0.1700

Data Index: 227753 (C4 (Web Text))

Max Activating Token Index: 835

Text #13

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0281. Min Act: -0.1700

Data Index: 479261 (C4 (Web Text))

Max Activating Token Index: 47

Text #14

Max Range: 3.1809. Min Range: -3.1809

Max Act: 2.9963. Min Act: -0.1700

Data Index: 930117 (C4 (Web Text))

Max Activating Token Index: 526

Text #15

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0164. Min Act: -0.1700

Data Index: 683524 (C4 (Web Text))

Max Activating Token Index: 592

Text #16

Max Range: 3.1809. Min Range: -3.1809

Max Act: 3.0038. Min Act: -0.1700

Data Index: 205210 (C4 (Web Text))

Max Activating Token Index: 473

Text #17

Max Range: 3.1809. Min Range: -3.1809

Max Act: 2.9988. Min Act: -0.1700

Data Index: 523520 (C4 (Web Text))

Max Activating Token Index: 425

Text #18

Max Range: 3.1809. Min Range: -3.1809

Max Act: 2.9787. Min Act: -0.1700

Data Index: 1228267 (C4 (Web Text))

Max Activating Token Index: 73

Text #19

Max Range: 3.1809. Min Range: -3.1809

Max Act: 2.9915. Min Act: -0.1700

Data Index: 1292138 (C4 (Web Text))

Max Activating Token Index: 848