Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1626 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.7098. Min Act: -0.1700

Data Index: 7839695 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.6621. Min Act: -0.1700

Data Index: 4305715 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.6289. Min Act: -0.1700

Data Index: 1516280 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5973. Min Act: -0.1700

Data Index: 479622 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5918. Min Act: -0.1700

Data Index: 7485453 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5802. Min Act: -0.1700

Data Index: 7062399 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5504. Min Act: -0.1700

Data Index: 1094368 (Open Web Text)

Max Activating Token Index: 600

Text #7

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5147. Min Act: -0.1700

Data Index: 8302014 (Open Web Text)

Max Activating Token Index: 636

Text #8

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5219. Min Act: -0.1700

Data Index: 8366381 (Open Web Text)

Max Activating Token Index: 877

Text #9

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5166. Min Act: -0.1700

Data Index: 1465589 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5025. Min Act: -0.1699

Data Index: 3455258 (Open Web Text)

Max Activating Token Index: 764

Text #11

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5047. Min Act: -0.1700

Data Index: 3965669 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5045. Min Act: -0.1700

Data Index: 8683098 (Open Web Text)

Max Activating Token Index: 463

Text #13

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.5133. Min Act: -0.1700

Data Index: 1458797 (Open Web Text)

Max Activating Token Index: 616

Text #14

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.4984. Min Act: -0.1700

Data Index: 3485954 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.4985. Min Act: -0.1699

Data Index: 5134071 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.4955. Min Act: -0.1700

Data Index: 7574246 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.4825. Min Act: -0.1699

Data Index: 5773881 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.4861. Min Act: -0.1699

Data Index: 1967879 (Open Web Text)

Max Activating Token Index: 320

Text #19

Max Range: 1.7098. Min Range: -1.7098

Max Act: 1.4918. Min Act: -0.1700

Data Index: 4938117 (Open Web Text)

Max Activating Token Index: 627