Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1721 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.6664. Min Act: -0.1700

Data Index: 3804521 (Open Web Text)

Max Activating Token Index: 936

Text #1

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4980. Min Act: -0.1700

Data Index: 7222849 (Open Web Text)

Max Activating Token Index: 266

Text #2

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4997. Min Act: -0.1700

Data Index: 6454453 (Open Web Text)

Max Activating Token Index: 504

Text #3

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.5305. Min Act: -0.1700

Data Index: 821281 (Open Web Text)

Max Activating Token Index: 787

Text #4

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4879. Min Act: -0.1700

Data Index: 2096426 (Open Web Text)

Max Activating Token Index: 689

Text #5

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.5008. Min Act: -0.1700

Data Index: 2269803 (Open Web Text)

Max Activating Token Index: 72

Text #6

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.5141. Min Act: -0.1700

Data Index: 6590430 (Open Web Text)

Max Activating Token Index: 850

Text #7

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.5148. Min Act: -0.1700

Data Index: 7174567 (Open Web Text)

Max Activating Token Index: 748

Text #8

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4882. Min Act: -0.1700

Data Index: 8429639 (Open Web Text)

Max Activating Token Index: 653

Text #9

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4834. Min Act: -0.1700

Data Index: 1530633 (Open Web Text)

Max Activating Token Index: 991

Text #10

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4508. Min Act: -0.1700

Data Index: 1494525 (Open Web Text)

Max Activating Token Index: 618

Text #11

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4819. Min Act: -0.1700

Data Index: 940005 (Open Web Text)

Max Activating Token Index: 951

Text #12

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4728. Min Act: -0.1700

Data Index: 6804034 (Open Web Text)

Max Activating Token Index: 773

Text #13

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4715. Min Act: -0.1700

Data Index: 4194665 (Open Web Text)

Max Activating Token Index: 699

Text #14

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4787. Min Act: -0.1700

Data Index: 5415179 (Open Web Text)

Max Activating Token Index: 639

Text #15

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4580. Min Act: -0.1700

Data Index: 3211298 (Open Web Text)

Max Activating Token Index: 766

Text #16

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4263. Min Act: -0.1700

Data Index: 4883019 (Open Web Text)

Max Activating Token Index: 703

Text #17

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4597. Min Act: -0.1700

Data Index: 1692607 (Open Web Text)

Max Activating Token Index: 483

Text #18

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4710. Min Act: -0.1700

Data Index: 139524 (Open Web Text)

Max Activating Token Index: 749

Text #19

Max Range: 3.6664. Min Range: -3.6664

Max Act: 3.4466. Min Act: -0.1700

Data Index: 1598892 (Open Web Text)

Max Activating Token Index: 82