Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 856 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1850. Min Act: -0.1700

Data Index: 4607261 (Open Web Text)

Max Activating Token Index: 405

Text #1

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1545. Min Act: -0.1700

Data Index: 7631752 (Open Web Text)

Max Activating Token Index: 300

Text #2

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1687. Min Act: -0.1700

Data Index: 8224552 (Open Web Text)

Max Activating Token Index: 486

Text #3

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1408. Min Act: -0.1700

Data Index: 3110593 (Open Web Text)

Max Activating Token Index: 353

Text #4

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1333. Min Act: -0.1700

Data Index: 8012663 (Open Web Text)

Max Activating Token Index: 89

Text #5

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1196. Min Act: -0.1700

Data Index: 6277338 (Open Web Text)

Max Activating Token Index: 933

Text #6

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1151. Min Act: -0.1700

Data Index: 1036054 (Open Web Text)

Max Activating Token Index: 85

Text #7

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.0977. Min Act: -0.1700

Data Index: 5098335 (Open Web Text)

Max Activating Token Index: 735

Text #8

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1020. Min Act: -0.1700

Data Index: 6425426 (Open Web Text)

Max Activating Token Index: 279

Text #9

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1027. Min Act: -0.1700

Data Index: 657420 (Open Web Text)

Max Activating Token Index: 780

Text #10

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1030. Min Act: -0.1700

Data Index: 8320591 (Open Web Text)

Max Activating Token Index: 888

Text #11

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1086. Min Act: -0.1700

Data Index: 3438947 (Open Web Text)

Max Activating Token Index: 958

Text #12

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1218. Min Act: -0.1700

Data Index: 4625673 (Open Web Text)

Max Activating Token Index: 935

Text #13

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.1029. Min Act: -0.1700

Data Index: 5491039 (Open Web Text)

Max Activating Token Index: 505

Text #14

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.0945. Min Act: -0.1700

Data Index: 5462097 (Open Web Text)

Max Activating Token Index: 340

Text #15

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.0866. Min Act: -0.1700

Data Index: 7838399 (Open Web Text)

Max Activating Token Index: 560

Text #16

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.0766. Min Act: -0.1700

Data Index: 7419965 (Open Web Text)

Max Activating Token Index: 525

Text #17

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.0662. Min Act: -0.1700

Data Index: 7978454 (Open Web Text)

Max Activating Token Index: 588

Text #18

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.0674. Min Act: -0.1700

Data Index: 5177508 (Open Web Text)

Max Activating Token Index: 92

Text #19

Max Range: 3.1850. Min Range: -3.1850

Max Act: 3.0529. Min Act: -0.1700

Data Index: 4695318 (Open Web Text)

Max Activating Token Index: 186