Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 276 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.1721. Min Act: -0.1700

Data Index: 5338383 (Open Web Text)

Max Activating Token Index: 448

Text #1

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0950. Min Act: -0.1700

Data Index: 2430814 (Open Web Text)

Max Activating Token Index: 845

Text #2

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0768. Min Act: -0.1700

Data Index: 6567508 (Open Web Text)

Max Activating Token Index: 149

Text #3

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0744. Min Act: -0.1700

Data Index: 6894510 (Open Web Text)

Max Activating Token Index: 361

Text #4

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0699. Min Act: -0.1700

Data Index: 2915965 (Open Web Text)

Max Activating Token Index: 587

Text #5

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0930. Min Act: -0.1700

Data Index: 7843578 (Open Web Text)

Max Activating Token Index: 989

Text #6

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0780. Min Act: -0.1700

Data Index: 3120077 (Open Web Text)

Max Activating Token Index: 993

Text #7

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0535. Min Act: -0.1700

Data Index: 4929828 (Open Web Text)

Max Activating Token Index: 904

Text #8

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0674. Min Act: -0.1700

Data Index: 573002 (Open Web Text)

Max Activating Token Index: 525

Text #9

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0243. Min Act: -0.1700

Data Index: 3811025 (Open Web Text)

Max Activating Token Index: 547

Text #10

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0401. Min Act: -0.1700

Data Index: 4445030 (Open Web Text)

Max Activating Token Index: 533

Text #11

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0251. Min Act: -0.1700

Data Index: 3261010 (Open Web Text)

Max Activating Token Index: 282

Text #12

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0079. Min Act: -0.1700

Data Index: 966413 (Open Web Text)

Max Activating Token Index: 375

Text #13

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0133. Min Act: -0.1700

Data Index: 777173 (Open Web Text)

Max Activating Token Index: 820

Text #14

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0181. Min Act: -0.1700

Data Index: 7412635 (Open Web Text)

Max Activating Token Index: 866

Text #15

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0116. Min Act: -0.1700

Data Index: 7095157 (Open Web Text)

Max Activating Token Index: 369

Text #16

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0123. Min Act: -0.1700

Data Index: 1347469 (Open Web Text)

Max Activating Token Index: 461

Text #17

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0400. Min Act: -0.1700

Data Index: 6506090 (Open Web Text)

Max Activating Token Index: 326

Text #18

Max Range: 4.1721. Min Range: -4.1721

Max Act: 4.0110. Min Act: -0.1700

Data Index: 4902801 (Open Web Text)

Max Activating Token Index: 979

Text #19

Max Range: 4.1721. Min Range: -4.1721

Max Act: 3.9866. Min Act: -0.1700

Data Index: 3356194 (Open Web Text)

Max Activating Token Index: 845