Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1705 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.4549. Min Act: -0.1700

Data Index: 5091054 (Open Web Text)

Max Activating Token Index: 220

Text #1

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.4203. Min Act: -0.1700

Data Index: 5630737 (Open Web Text)

Max Activating Token Index: 215

Text #2

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.3832. Min Act: -0.1700

Data Index: 8787311 (Open Web Text)

Max Activating Token Index: 704

Text #3

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.3224. Min Act: -0.1700

Data Index: 292959 (Open Web Text)

Max Activating Token Index: 109

Text #4

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2832. Min Act: -0.1700

Data Index: 1127245 (Open Web Text)

Max Activating Token Index: 254

Text #5

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.3138. Min Act: -0.1700

Data Index: 629837 (Open Web Text)

Max Activating Token Index: 893

Text #6

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2758. Min Act: -0.1700

Data Index: 5614642 (Open Web Text)

Max Activating Token Index: 117

Text #7

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.3043. Min Act: -0.1700

Data Index: 7696000 (Open Web Text)

Max Activating Token Index: 682

Text #8

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2596. Min Act: -0.1700

Data Index: 2085822 (Open Web Text)

Max Activating Token Index: 408

Text #9

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2750. Min Act: -0.1700

Data Index: 8807449 (Open Web Text)

Max Activating Token Index: 19

Text #10

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2660. Min Act: -0.1700

Data Index: 1966747 (Open Web Text)

Max Activating Token Index: 241

Text #11

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2685. Min Act: -0.1700

Data Index: 3137532 (Open Web Text)

Max Activating Token Index: 542

Text #12

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2679. Min Act: -0.1700

Data Index: 3161246 (Open Web Text)

Max Activating Token Index: 863

Text #13

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2895. Min Act: -0.1700

Data Index: 4205430 (Open Web Text)

Max Activating Token Index: 430

Text #14

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2709. Min Act: -0.1700

Data Index: 6958793 (Open Web Text)

Max Activating Token Index: 401

Text #15

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2513. Min Act: -0.1700

Data Index: 5110384 (Open Web Text)

Max Activating Token Index: 187

Text #16

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2697. Min Act: -0.1700

Data Index: 5938196 (Open Web Text)

Max Activating Token Index: 646

Text #17

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2843. Min Act: -0.1700

Data Index: 6323966 (Open Web Text)

Max Activating Token Index: 537

Text #18

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2563. Min Act: -0.1700

Data Index: 6442400 (Open Web Text)

Max Activating Token Index: 406

Text #19

Max Range: 4.4549. Min Range: -4.4549

Max Act: 4.2349. Min Act: -0.1700

Data Index: 1837749 (Open Web Text)

Max Activating Token Index: 642