Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1418 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.6403. Min Act: -0.1700

Data Index: 5829889 (Open Web Text)

Max Activating Token Index: 640

Text #1

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.5146. Min Act: -0.1700

Data Index: 4028985 (Open Web Text)

Max Activating Token Index: 687

Text #2

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.3440. Min Act: -0.1700

Data Index: 230674 (Open Web Text)

Max Activating Token Index: 668

Text #3

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.1253. Min Act: -0.1700

Data Index: 7534367 (Open Web Text)

Max Activating Token Index: 619

Text #4

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.1904. Min Act: -0.1700

Data Index: 5422216 (Open Web Text)

Max Activating Token Index: 628

Text #5

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.1777. Min Act: -0.1700

Data Index: 4270816 (Open Web Text)

Max Activating Token Index: 611

Text #6

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.1868. Min Act: -0.1700

Data Index: 5370992 (Open Web Text)

Max Activating Token Index: 129

Text #7

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.1000. Min Act: -0.1700

Data Index: 857892 (Open Web Text)

Max Activating Token Index: 649

Text #8

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.1050. Min Act: -0.1700

Data Index: 6065122 (Open Web Text)

Max Activating Token Index: 678

Text #9

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.1164. Min Act: -0.1700

Data Index: 5393248 (Open Web Text)

Max Activating Token Index: 633

Text #10

Max Range: 6.6403. Min Range: -6.6403

Max Act: 6.0426. Min Act: -0.1700

Data Index: 8200547 (Open Web Text)

Max Activating Token Index: 659

Text #11

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.9812. Min Act: -0.1700

Data Index: 531504 (Open Web Text)

Max Activating Token Index: 661

Text #12

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.9567. Min Act: -0.1700

Data Index: 6528477 (Open Web Text)

Max Activating Token Index: 586

Text #13

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.9168. Min Act: -0.1700

Data Index: 929573 (Open Web Text)

Max Activating Token Index: 659

Text #14

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.9449. Min Act: -0.1700

Data Index: 5452728 (Open Web Text)

Max Activating Token Index: 656

Text #15

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.9562. Min Act: -0.1700

Data Index: 2805132 (Open Web Text)

Max Activating Token Index: 654

Text #16

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.8808. Min Act: -0.1700

Data Index: 1074720 (Open Web Text)

Max Activating Token Index: 141

Text #17

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.8496. Min Act: -0.1700

Data Index: 7338887 (Open Web Text)

Max Activating Token Index: 129

Text #18

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.9382. Min Act: -0.1700

Data Index: 2245140 (Open Web Text)

Max Activating Token Index: 625

Text #19

Max Range: 6.6403. Min Range: -6.6403

Max Act: 5.8569. Min Act: -0.1700

Data Index: 5289425 (Open Web Text)

Max Activating Token Index: 663