Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2019 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 9.0036. Min Range: -9.0036

Max Act: 9.0036. Min Act: -0.1700

Data Index: 948975 (C4 (Web Text))

Max Activating Token Index: 598

Text #1

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.7191. Min Act: -0.1700

Data Index: 133000 (C4 (Web Text))

Max Activating Token Index: 193

Text #2

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.4208. Min Act: -0.1700

Data Index: 661825 (C4 (Web Text))

Max Activating Token Index: 130

Text #3

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.3423. Min Act: -0.1700

Data Index: 934296 (C4 (Web Text))

Max Activating Token Index: 349

Text #4

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2584. Min Act: -0.1700

Data Index: 1176808 (C4 (Web Text))

Max Activating Token Index: 283

Text #5

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.3425. Min Act: -0.1700

Data Index: 752059 (C4 (Web Text))

Max Activating Token Index: 412

Text #6

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.3171. Min Act: -0.1700

Data Index: 842293 (C4 (Web Text))

Max Activating Token Index: 157

Text #7

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2679. Min Act: -0.1700

Data Index: 193078 (C4 (Web Text))

Max Activating Token Index: 893

Text #8

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2756. Min Act: -0.1700

Data Index: 649649 (C4 (Web Text))

Max Activating Token Index: 868

Text #9

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2978. Min Act: -0.1700

Data Index: 370754 (C4 (Web Text))

Max Activating Token Index: 134

Text #10

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2755. Min Act: -0.1700

Data Index: 366580 (C4 (Web Text))

Max Activating Token Index: 324

Text #11

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.1921. Min Act: -0.1700

Data Index: 916209 (C4 (Web Text))

Max Activating Token Index: 849

Text #12

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.1380. Min Act: -0.1700

Data Index: 1219828 (C4 (Web Text))

Max Activating Token Index: 405

Text #13

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2318. Min Act: -0.1700

Data Index: 909108 (C4 (Web Text))

Max Activating Token Index: 399

Text #14

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2105. Min Act: -0.1700

Data Index: 124469 (C4 (Web Text))

Max Activating Token Index: 341

Text #15

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.0977. Min Act: -0.1700

Data Index: 772807 (C4 (Web Text))

Max Activating Token Index: 76

Text #16

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.1819. Min Act: -0.1700

Data Index: 1344876 (C4 (Web Text))

Max Activating Token Index: 818

Text #17

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.1277. Min Act: -0.1700

Data Index: 801975 (C4 (Web Text))

Max Activating Token Index: 781

Text #18

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.2099. Min Act: -0.1700

Data Index: 102273 (C4 (Web Text))

Max Activating Token Index: 956

Text #19

Max Range: 9.0036. Min Range: -9.0036

Max Act: 8.0486. Min Act: -0.1700

Data Index: 462519 (C4 (Web Text))

Max Activating Token Index: 458