Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 680 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0740. Min Act: -0.1700

Data Index: 3634352 (Open Web Text)

Max Activating Token Index: 737

Text #1

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0698. Min Act: -0.1700

Data Index: 8670798 (Open Web Text)

Max Activating Token Index: 417

Text #2

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0428. Min Act: -0.1700

Data Index: 3682307 (Open Web Text)

Max Activating Token Index: 14

Text #3

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0030. Min Act: -0.1700

Data Index: 6263633 (Open Web Text)

Max Activating Token Index: 530

Text #4

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0253. Min Act: -0.1700

Data Index: 2157069 (Open Web Text)

Max Activating Token Index: 20

Text #5

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0287. Min Act: -0.1700

Data Index: 1206374 (Open Web Text)

Max Activating Token Index: 449

Text #6

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0043. Min Act: -0.1700

Data Index: 6194657 (Open Web Text)

Max Activating Token Index: 164

Text #7

Max Range: 4.0740. Min Range: -4.0740

Max Act: 4.0109. Min Act: -0.1700

Data Index: 6373085 (Open Web Text)

Max Activating Token Index: 412

Text #8

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.9924. Min Act: -0.1700

Data Index: 6482642 (Open Web Text)

Max Activating Token Index: 309

Text #9

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.9977. Min Act: -0.1700

Data Index: 1296072 (Open Web Text)

Max Activating Token Index: 584

Text #10

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.9546. Min Act: -0.1700

Data Index: 6961404 (Open Web Text)

Max Activating Token Index: 53

Text #11

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.9499. Min Act: -0.1700

Data Index: 6718424 (Open Web Text)

Max Activating Token Index: 956

Text #12

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.9491. Min Act: -0.1700

Data Index: 891416 (Open Web Text)

Max Activating Token Index: 8

Text #13

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.9300. Min Act: -0.1700

Data Index: 8071985 (Open Web Text)

Max Activating Token Index: 629

Text #14

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.9256. Min Act: -0.1700

Data Index: 6677640 (Open Web Text)

Max Activating Token Index: 433

Text #15

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.8987. Min Act: -0.1700

Data Index: 5418870 (Open Web Text)

Max Activating Token Index: 705

Text #16

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.8730. Min Act: -0.1700

Data Index: 7023663 (Open Web Text)

Max Activating Token Index: 454

Text #17

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.8882. Min Act: -0.1700

Data Index: 7472365 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.8931. Min Act: -0.1700

Data Index: 7659269 (Open Web Text)

Max Activating Token Index: 479

Text #19

Max Range: 4.0740. Min Range: -4.0740

Max Act: 3.8585. Min Act: -0.1700

Data Index: 5619834 (Open Web Text)

Max Activating Token Index: 886