Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 286 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.3754. Min Act: -0.1700

Data Index: 7807967 (Open Web Text)

Max Activating Token Index: 402

Text #1

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.3013. Min Act: -0.1700

Data Index: 5633720 (Open Web Text)

Max Activating Token Index: 382

Text #2

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.2909. Min Act: -0.1700

Data Index: 6263901 (Open Web Text)

Max Activating Token Index: 776

Text #3

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.2785. Min Act: -0.1700

Data Index: 4378675 (Open Web Text)

Max Activating Token Index: 224

Text #4

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.2409. Min Act: -0.1700

Data Index: 7447386 (Open Web Text)

Max Activating Token Index: 484

Text #5

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.2519. Min Act: -0.1700

Data Index: 7447388 (Open Web Text)

Max Activating Token Index: 411

Text #6

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.2236. Min Act: -0.1700

Data Index: 426588 (Open Web Text)

Max Activating Token Index: 82

Text #7

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1916. Min Act: -0.1700

Data Index: 2092426 (Open Web Text)

Max Activating Token Index: 568

Text #8

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.2118. Min Act: -0.1700

Data Index: 1543439 (Open Web Text)

Max Activating Token Index: 1020

Text #9

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.2049. Min Act: -0.1700

Data Index: 2829838 (Open Web Text)

Max Activating Token Index: 363

Text #10

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1448. Min Act: -0.1700

Data Index: 5536087 (Open Web Text)

Max Activating Token Index: 871

Text #11

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1577. Min Act: -0.1700

Data Index: 2185383 (Open Web Text)

Max Activating Token Index: 675

Text #12

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1616. Min Act: -0.1700

Data Index: 2097995 (Open Web Text)

Max Activating Token Index: 185

Text #13

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1288. Min Act: -0.1700

Data Index: 6485839 (Open Web Text)

Max Activating Token Index: 78

Text #14

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1103. Min Act: -0.1700

Data Index: 5409761 (Open Web Text)

Max Activating Token Index: 96

Text #15

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1419. Min Act: -0.1700

Data Index: 4513402 (Open Web Text)

Max Activating Token Index: 467

Text #16

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1173. Min Act: -0.1700

Data Index: 425347 (Open Web Text)

Max Activating Token Index: 284

Text #17

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.0902. Min Act: -0.1700

Data Index: 3595321 (Open Web Text)

Max Activating Token Index: 577

Text #18

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1570. Min Act: -0.1700

Data Index: 4330435 (Open Web Text)

Max Activating Token Index: 766

Text #19

Max Range: 4.3754. Min Range: -4.3754

Max Act: 4.1408. Min Act: -0.1700

Data Index: 6101048 (Open Web Text)

Max Activating Token Index: 395