Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1875 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3584. Min Act: -0.1664

Data Index: 37800 (Open Web Text)

Max Activating Token Index: 902

Text #1

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3547. Min Act: -0.1699

Data Index: 7398924 (Open Web Text)

Max Activating Token Index: 320

Text #2

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3458. Min Act: -0.1684

Data Index: 5441117 (Open Web Text)

Max Activating Token Index: 434

Text #3

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3430. Min Act: -0.1681

Data Index: 3451247 (Open Web Text)

Max Activating Token Index: 760

Text #4

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3312. Min Act: -0.1693

Data Index: 5916389 (Open Web Text)

Max Activating Token Index: 236

Text #5

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3369. Min Act: -0.1700

Data Index: 1408470 (Open Web Text)

Max Activating Token Index: 62

Text #6

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3267. Min Act: -0.1678

Data Index: 6871275 (Open Web Text)

Max Activating Token Index: 616

Text #7

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3149. Min Act: -0.1700

Data Index: 16678 (Open Web Text)

Max Activating Token Index: 753

Text #8

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3100. Min Act: -0.1693

Data Index: 7982250 (Open Web Text)

Max Activating Token Index: 337

Text #9

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3119. Min Act: -0.1680

Data Index: 8541215 (Open Web Text)

Max Activating Token Index: 506

Text #10

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3069. Min Act: -0.1698

Data Index: 1481223 (Open Web Text)

Max Activating Token Index: 40

Text #11

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3164. Min Act: -0.1699

Data Index: 3535335 (Open Web Text)

Max Activating Token Index: 300

Text #12

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3081. Min Act: -0.1700

Data Index: 1380590 (Open Web Text)

Max Activating Token Index: 439

Text #13

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3097. Min Act: -0.1688

Data Index: 2799322 (Open Web Text)

Max Activating Token Index: 704

Text #14

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3061. Min Act: -0.1687

Data Index: 4270023 (Open Web Text)

Max Activating Token Index: 714

Text #15

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.2980. Min Act: -0.1700

Data Index: 4433743 (Open Web Text)

Max Activating Token Index: 256

Text #16

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3023. Min Act: -0.1697

Data Index: 6979349 (Open Web Text)

Max Activating Token Index: 508

Text #17

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.2984. Min Act: -0.1694

Data Index: 6979363 (Open Web Text)

Max Activating Token Index: 632

Text #18

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.2949. Min Act: -0.1700

Data Index: 3646529 (Open Web Text)

Max Activating Token Index: 322

Text #19

Max Range: 1.3584. Min Range: -1.3584

Max Act: 1.3018. Min Act: -0.1691

Data Index: 4250578 (Open Web Text)

Max Activating Token Index: 336