Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 561 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0745. Min Act: -0.1700

Data Index: 1539710 (Open Web Text)

Max Activating Token Index: 219

Text #1

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0746. Min Act: -0.1700

Data Index: 60709 (Open Web Text)

Max Activating Token Index: 675

Text #2

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0554. Min Act: -0.1700

Data Index: 6123422 (Open Web Text)

Max Activating Token Index: 423

Text #3

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0709. Min Act: -0.1700

Data Index: 1832743 (Open Web Text)

Max Activating Token Index: 439

Text #4

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0638. Min Act: -0.1700

Data Index: 8557818 (Open Web Text)

Max Activating Token Index: 895

Text #5

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0319. Min Act: -0.1700

Data Index: 1435040 (Open Web Text)

Max Activating Token Index: 319

Text #6

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0161. Min Act: -0.1700

Data Index: 1661964 (Open Web Text)

Max Activating Token Index: 746

Text #7

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9868. Min Act: -0.1700

Data Index: 6949962 (Open Web Text)

Max Activating Token Index: 860

Text #8

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0118. Min Act: -0.1700

Data Index: 4772687 (Open Web Text)

Max Activating Token Index: 574

Text #9

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0220. Min Act: -0.1700

Data Index: 230395 (Open Web Text)

Max Activating Token Index: 614

Text #10

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0211. Min Act: -0.1700

Data Index: 6095465 (Open Web Text)

Max Activating Token Index: 849

Text #11

Max Range: 4.0746. Min Range: -4.0746

Max Act: 4.0161. Min Act: -0.1700

Data Index: 3130711 (Open Web Text)

Max Activating Token Index: 455

Text #12

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9815. Min Act: -0.1700

Data Index: 6829307 (Open Web Text)

Max Activating Token Index: 347

Text #13

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9908. Min Act: -0.1700

Data Index: 4212162 (Open Web Text)

Max Activating Token Index: 491

Text #14

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9926. Min Act: -0.1700

Data Index: 390550 (Open Web Text)

Max Activating Token Index: 381

Text #15

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9766. Min Act: -0.1700

Data Index: 3413198 (Open Web Text)

Max Activating Token Index: 926

Text #16

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9770. Min Act: -0.1700

Data Index: 3755712 (Open Web Text)

Max Activating Token Index: 499

Text #17

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9258. Min Act: -0.1700

Data Index: 6178198 (Open Web Text)

Max Activating Token Index: 392

Text #18

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9456. Min Act: -0.1700

Data Index: 4038161 (Open Web Text)

Max Activating Token Index: 696

Text #19

Max Range: 4.0746. Min Range: -4.0746

Max Act: 3.9566. Min Act: -0.1700

Data Index: 507334 (Open Web Text)

Max Activating Token Index: 427