Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1055 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.2800. Min Act: -0.1700

Data Index: 2449355 (Open Web Text)

Max Activating Token Index: 343

Text #1

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.2228. Min Act: -0.1700

Data Index: 162500 (Open Web Text)

Max Activating Token Index: 38

Text #2

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.1109. Min Act: -0.1700

Data Index: 1086966 (Open Web Text)

Max Activating Token Index: 386

Text #3

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0621. Min Act: -0.1700

Data Index: 7827479 (Open Web Text)

Max Activating Token Index: 463

Text #4

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0813. Min Act: -0.1700

Data Index: 5746126 (Open Web Text)

Max Activating Token Index: 950

Text #5

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0668. Min Act: -0.1700

Data Index: 7680720 (Open Web Text)

Max Activating Token Index: 408

Text #6

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0850. Min Act: -0.1700

Data Index: 2725922 (Open Web Text)

Max Activating Token Index: 1005

Text #7

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0436. Min Act: -0.1700

Data Index: 8220169 (Open Web Text)

Max Activating Token Index: 653

Text #8

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0569. Min Act: -0.1700

Data Index: 2058884 (Open Web Text)

Max Activating Token Index: 440

Text #9

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0504. Min Act: -0.1700

Data Index: 8249637 (Open Web Text)

Max Activating Token Index: 926

Text #10

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0594. Min Act: -0.1700

Data Index: 4999448 (Open Web Text)

Max Activating Token Index: 809

Text #11

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0399. Min Act: -0.1700

Data Index: 8522748 (Open Web Text)

Max Activating Token Index: 380

Text #12

Max Range: 3.2800. Min Range: -3.2800

Max Act: 3.0102. Min Act: -0.1700

Data Index: 5950057 (Open Web Text)

Max Activating Token Index: 617

Text #13

Max Range: 3.2800. Min Range: -3.2800

Max Act: 2.9915. Min Act: -0.1700

Data Index: 3502488 (Open Web Text)

Max Activating Token Index: 917

Text #14

Max Range: 3.2800. Min Range: -3.2800

Max Act: 2.9814. Min Act: -0.1700

Data Index: 4877540 (Open Web Text)

Max Activating Token Index: 61

Text #15

Max Range: 3.2800. Min Range: -3.2800

Max Act: 2.9835. Min Act: -0.1700

Data Index: 6189372 (Open Web Text)

Max Activating Token Index: 672

Text #16

Max Range: 3.2800. Min Range: -3.2800

Max Act: 2.9692. Min Act: -0.1700

Data Index: 2676994 (Open Web Text)

Max Activating Token Index: 348

Text #17

Max Range: 3.2800. Min Range: -3.2800

Max Act: 2.9625. Min Act: -0.1700

Data Index: 2894958 (Open Web Text)

Max Activating Token Index: 677

Text #18

Max Range: 3.2800. Min Range: -3.2800

Max Act: 2.9422. Min Act: -0.1700

Data Index: 2629097 (Open Web Text)

Max Activating Token Index: 150

Text #19

Max Range: 3.2800. Min Range: -3.2800

Max Act: 2.9448. Min Act: -0.1700

Data Index: 2524062 (Open Web Text)

Max Activating Token Index: 692