Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 678 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.3008. Min Act: -0.1700

Data Index: 8647678 (Open Web Text)

Max Activating Token Index: 526

Text #1

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.2229. Min Act: -0.1700

Data Index: 7695786 (Open Web Text)

Max Activating Token Index: 216

Text #2

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.2034. Min Act: -0.1700

Data Index: 6796805 (Open Web Text)

Max Activating Token Index: 223

Text #3

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.1719. Min Act: -0.1700

Data Index: 6652338 (Open Web Text)

Max Activating Token Index: 465

Text #4

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.1204. Min Act: -0.1700

Data Index: 4032746 (Open Web Text)

Max Activating Token Index: 37

Text #5

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.1226. Min Act: -0.1700

Data Index: 5195914 (Open Web Text)

Max Activating Token Index: 981

Text #6

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.1027. Min Act: -0.1700

Data Index: 5569650 (Open Web Text)

Max Activating Token Index: 22

Text #7

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.1182. Min Act: -0.1700

Data Index: 7904059 (Open Web Text)

Max Activating Token Index: 847

Text #8

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0303. Min Act: -0.1700

Data Index: 5716233 (Open Web Text)

Max Activating Token Index: 311

Text #9

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0427. Min Act: -0.1700

Data Index: 8166739 (Open Web Text)

Max Activating Token Index: 519

Text #10

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0369. Min Act: -0.1700

Data Index: 7157859 (Open Web Text)

Max Activating Token Index: 600

Text #11

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0783. Min Act: -0.1700

Data Index: 6909130 (Open Web Text)

Max Activating Token Index: 571

Text #12

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.1103. Min Act: -0.1700

Data Index: 8429017 (Open Web Text)

Max Activating Token Index: 585

Text #13

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0944. Min Act: -0.1700

Data Index: 6331876 (Open Web Text)

Max Activating Token Index: 275

Text #14

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0689. Min Act: -0.1700

Data Index: 2378397 (Open Web Text)

Max Activating Token Index: 561

Text #15

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0486. Min Act: -0.1700

Data Index: 8056650 (Open Web Text)

Max Activating Token Index: 90

Text #16

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0300. Min Act: -0.1700

Data Index: 700243 (Open Web Text)

Max Activating Token Index: 391

Text #17

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0466. Min Act: -0.1700

Data Index: 7614336 (Open Web Text)

Max Activating Token Index: 298

Text #18

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0376. Min Act: -0.1700

Data Index: 3258943 (Open Web Text)

Max Activating Token Index: 784

Text #19

Max Range: 5.3008. Min Range: -5.3008

Max Act: 5.0393. Min Act: -0.1700

Data Index: 1887581 (Open Web Text)

Max Activating Token Index: 404