Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1912 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.4658. Min Act: -0.1700

Data Index: 8824293 (Open Web Text)

Max Activating Token Index: 603

Text #1

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.4216. Min Act: -0.1700

Data Index: 5320634 (Open Web Text)

Max Activating Token Index: 390

Text #2

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.4112. Min Act: -0.1700

Data Index: 8079130 (Open Web Text)

Max Activating Token Index: 539

Text #3

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3989. Min Act: -0.1700

Data Index: 5520209 (Open Web Text)

Max Activating Token Index: 760

Text #4

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.4180. Min Act: -0.1700

Data Index: 1709208 (Open Web Text)

Max Activating Token Index: 887

Text #5

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3659. Min Act: -0.1700

Data Index: 7157716 (Open Web Text)

Max Activating Token Index: 355

Text #6

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3436. Min Act: -0.1700

Data Index: 4170866 (Open Web Text)

Max Activating Token Index: 431

Text #7

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3653. Min Act: -0.1700

Data Index: 4757885 (Open Web Text)

Max Activating Token Index: 656

Text #8

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3689. Min Act: -0.1700

Data Index: 5373292 (Open Web Text)

Max Activating Token Index: 717

Text #9

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3109. Min Act: -0.1700

Data Index: 5934695 (Open Web Text)

Max Activating Token Index: 579

Text #10

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3180. Min Act: -0.1700

Data Index: 7716611 (Open Web Text)

Max Activating Token Index: 196

Text #11

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3125. Min Act: -0.1700

Data Index: 8302358 (Open Web Text)

Max Activating Token Index: 601

Text #12

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3260. Min Act: -0.1700

Data Index: 1319427 (Open Web Text)

Max Activating Token Index: 143

Text #13

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3069. Min Act: -0.1700

Data Index: 4367750 (Open Web Text)

Max Activating Token Index: 45

Text #14

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3004. Min Act: -0.1700

Data Index: 5520207 (Open Web Text)

Max Activating Token Index: 609

Text #15

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.2966. Min Act: -0.1700

Data Index: 3162147 (Open Web Text)

Max Activating Token Index: 705

Text #16

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.2963. Min Act: -0.1700

Data Index: 5283983 (Open Web Text)

Max Activating Token Index: 802

Text #17

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.2906. Min Act: -0.1700

Data Index: 2513066 (Open Web Text)

Max Activating Token Index: 342

Text #18

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.3075. Min Act: -0.1700

Data Index: 440560 (Open Web Text)

Max Activating Token Index: 198

Text #19

Max Range: 5.4658. Min Range: -5.4658

Max Act: 5.2549. Min Act: -0.1700

Data Index: 5143679 (Open Web Text)

Max Activating Token Index: 986