Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2086 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.3707. Min Act: -0.1700

Data Index: 469559 (Open Web Text)

Max Activating Token Index: 133

Text #1

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.3723. Min Act: -0.1700

Data Index: 1050339 (Open Web Text)

Max Activating Token Index: 850

Text #2

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2973. Min Act: -0.1700

Data Index: 805258 (Open Web Text)

Max Activating Token Index: 860

Text #3

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.3455. Min Act: -0.1700

Data Index: 865793 (Open Web Text)

Max Activating Token Index: 160

Text #4

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2697. Min Act: -0.1700

Data Index: 1454851 (Open Web Text)

Max Activating Token Index: 406

Text #5

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2520. Min Act: -0.1700

Data Index: 5155357 (Open Web Text)

Max Activating Token Index: 219

Text #6

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2614. Min Act: -0.1700

Data Index: 6723746 (Open Web Text)

Max Activating Token Index: 685

Text #7

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2458. Min Act: -0.1700

Data Index: 812707 (Open Web Text)

Max Activating Token Index: 829

Text #8

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2663. Min Act: -0.1700

Data Index: 8597630 (Open Web Text)

Max Activating Token Index: 716

Text #9

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2247. Min Act: -0.1700

Data Index: 4596371 (Open Web Text)

Max Activating Token Index: 125

Text #10

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2405. Min Act: -0.1700

Data Index: 2859373 (Open Web Text)

Max Activating Token Index: 984

Text #11

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2391. Min Act: -0.1700

Data Index: 2205622 (Open Web Text)

Max Activating Token Index: 710

Text #12

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.2117. Min Act: -0.1700

Data Index: 7377620 (Open Web Text)

Max Activating Token Index: 869

Text #13

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.1620. Min Act: -0.1700

Data Index: 6388044 (Open Web Text)

Max Activating Token Index: 778

Text #14

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.1772. Min Act: -0.1700

Data Index: 8553903 (Open Web Text)

Max Activating Token Index: 55

Text #15

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.1850. Min Act: -0.1700

Data Index: 549868 (Open Web Text)

Max Activating Token Index: 207

Text #16

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.1503. Min Act: -0.1700

Data Index: 5947871 (Open Web Text)

Max Activating Token Index: 599

Text #17

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.1853. Min Act: -0.1700

Data Index: 2347762 (Open Web Text)

Max Activating Token Index: 621

Text #18

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.1242. Min Act: -0.1700

Data Index: 533412 (Open Web Text)

Max Activating Token Index: 933

Text #19

Max Range: 4.3723. Min Range: -4.3723

Max Act: 4.1444. Min Act: -0.1700

Data Index: 394833 (Open Web Text)

Max Activating Token Index: 458