Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 457 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.4900. Min Act: -0.1700

Data Index: 5321885 (Open Web Text)

Max Activating Token Index: 344

Text #1

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.3730. Min Act: -0.1700

Data Index: 8631652 (Open Web Text)

Max Activating Token Index: 833

Text #2

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.3347. Min Act: -0.1700

Data Index: 5136524 (Open Web Text)

Max Activating Token Index: 289

Text #3

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.3445. Min Act: -0.1700

Data Index: 961687 (Open Web Text)

Max Activating Token Index: 742

Text #4

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.3090. Min Act: -0.1700

Data Index: 4281877 (Open Web Text)

Max Activating Token Index: 757

Text #5

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.2500. Min Act: -0.1700

Data Index: 2924189 (Open Web Text)

Max Activating Token Index: 715

Text #6

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.2481. Min Act: -0.1700

Data Index: 6602482 (Open Web Text)

Max Activating Token Index: 953

Text #7

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.2458. Min Act: -0.1700

Data Index: 49014 (Open Web Text)

Max Activating Token Index: 454

Text #8

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.2226. Min Act: -0.1700

Data Index: 56800 (Open Web Text)

Max Activating Token Index: 381

Text #9

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1998. Min Act: -0.1700

Data Index: 3706125 (Open Web Text)

Max Activating Token Index: 878

Text #10

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.2011. Min Act: -0.1700

Data Index: 15128 (Open Web Text)

Max Activating Token Index: 915

Text #11

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.2202. Min Act: -0.1700

Data Index: 1070576 (Open Web Text)

Max Activating Token Index: 475

Text #12

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1737. Min Act: -0.1700

Data Index: 8381219 (Open Web Text)

Max Activating Token Index: 586

Text #13

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1816. Min Act: -0.1700

Data Index: 7074024 (Open Web Text)

Max Activating Token Index: 600

Text #14

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.2084. Min Act: -0.1700

Data Index: 2483068 (Open Web Text)

Max Activating Token Index: 803

Text #15

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1685. Min Act: -0.1700

Data Index: 6907231 (Open Web Text)

Max Activating Token Index: 306

Text #16

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1652. Min Act: -0.1700

Data Index: 3620831 (Open Web Text)

Max Activating Token Index: 283

Text #17

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1790. Min Act: -0.1700

Data Index: 5510437 (Open Web Text)

Max Activating Token Index: 865

Text #18

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1874. Min Act: -0.1700

Data Index: 3022376 (Open Web Text)

Max Activating Token Index: 368

Text #19

Max Range: 3.4900. Min Range: -3.4900

Max Act: 3.1832. Min Act: -0.1700

Data Index: 6442811 (Open Web Text)

Max Activating Token Index: 939