Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1522 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.5681. Min Act: -0.1700

Data Index: 140637 (Open Web Text)

Max Activating Token Index: 300

Text #1

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.5568. Min Act: -0.1700

Data Index: 1825696 (Open Web Text)

Max Activating Token Index: 82

Text #2

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.4242. Min Act: -0.1700

Data Index: 63390 (Open Web Text)

Max Activating Token Index: 852

Text #3

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.3765. Min Act: -0.1700

Data Index: 4331998 (Open Web Text)

Max Activating Token Index: 889

Text #4

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.3407. Min Act: -0.1700

Data Index: 7887484 (Open Web Text)

Max Activating Token Index: 427

Text #5

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.3290. Min Act: -0.1700

Data Index: 5584011 (Open Web Text)

Max Activating Token Index: 243

Text #6

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.3183. Min Act: -0.1700

Data Index: 7695619 (Open Web Text)

Max Activating Token Index: 44

Text #7

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.3201. Min Act: -0.1700

Data Index: 3199706 (Open Web Text)

Max Activating Token Index: 165

Text #8

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2859. Min Act: -0.1700

Data Index: 5492822 (Open Web Text)

Max Activating Token Index: 639

Text #9

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.3530. Min Act: -0.1700

Data Index: 7200058 (Open Web Text)

Max Activating Token Index: 1002

Text #10

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2909. Min Act: -0.1700

Data Index: 2589707 (Open Web Text)

Max Activating Token Index: 836

Text #11

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2846. Min Act: -0.1700

Data Index: 3424752 (Open Web Text)

Max Activating Token Index: 1007

Text #12

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2356. Min Act: -0.1700

Data Index: 2359827 (Open Web Text)

Max Activating Token Index: 947

Text #13

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2723. Min Act: -0.1700

Data Index: 900627 (Open Web Text)

Max Activating Token Index: 508

Text #14

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2712. Min Act: -0.1700

Data Index: 3198099 (Open Web Text)

Max Activating Token Index: 354

Text #15

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2367. Min Act: -0.1700

Data Index: 5052837 (Open Web Text)

Max Activating Token Index: 836

Text #16

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2359. Min Act: -0.1700

Data Index: 2069842 (Open Web Text)

Max Activating Token Index: 249

Text #17

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2271. Min Act: -0.1700

Data Index: 5357167 (Open Web Text)

Max Activating Token Index: 207

Text #18

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2311. Min Act: -0.1700

Data Index: 5237235 (Open Web Text)

Max Activating Token Index: 596

Text #19

Max Range: 4.5681. Min Range: -4.5681

Max Act: 4.2421. Min Act: -0.1700

Data Index: 2851954 (Open Web Text)

Max Activating Token Index: 240