Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1382 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.7466. Min Act: -0.1700

Data Index: 4224694 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.6933. Min Act: -0.1700

Data Index: 6478552 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.6172. Min Act: -0.1700

Data Index: 5469734 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5864. Min Act: -0.1700

Data Index: 7666506 (Open Web Text)

Max Activating Token Index: 199

Text #4

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5871. Min Act: -0.1700

Data Index: 1744372 (Open Web Text)

Max Activating Token Index: 582

Text #5

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5768. Min Act: -0.1700

Data Index: 7745195 (Open Web Text)

Max Activating Token Index: 331

Text #6

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5501. Min Act: -0.1700

Data Index: 2437018 (Open Web Text)

Max Activating Token Index: 11

Text #7

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5551. Min Act: -0.1700

Data Index: 6565866 (Open Web Text)

Max Activating Token Index: 167

Text #8

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5391. Min Act: -0.1700

Data Index: 6510773 (Open Web Text)

Max Activating Token Index: 905

Text #9

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5336. Min Act: -0.1700

Data Index: 7868990 (Open Web Text)

Max Activating Token Index: 904

Text #10

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5269. Min Act: -0.1700

Data Index: 4295695 (Open Web Text)

Max Activating Token Index: 241

Text #11

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5310. Min Act: -0.1700

Data Index: 6225784 (Open Web Text)

Max Activating Token Index: 796

Text #12

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5258. Min Act: -0.1700

Data Index: 7970771 (Open Web Text)

Max Activating Token Index: 687

Text #13

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.4961. Min Act: -0.1700

Data Index: 6648187 (Open Web Text)

Max Activating Token Index: 539

Text #14

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5208. Min Act: -0.1700

Data Index: 2592890 (Open Web Text)

Max Activating Token Index: 799

Text #15

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5162. Min Act: -0.1700

Data Index: 6570457 (Open Web Text)

Max Activating Token Index: 638

Text #16

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5157. Min Act: -0.1700

Data Index: 8218689 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.5157. Min Act: -0.1700

Data Index: 7633805 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.4935. Min Act: -0.1700

Data Index: 734410 (Open Web Text)

Max Activating Token Index: 520

Text #19

Max Range: 2.7466. Min Range: -2.7466

Max Act: 2.4890. Min Act: -0.1700

Data Index: 4557294 (Open Web Text)

Max Activating Token Index: 2