Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1134 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2347. Min Range: -5.2347

Max Act: 5.2347. Min Act: -0.1700

Data Index: 4104784 (Open Web Text)

Max Activating Token Index: 654

Text #1

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.9258. Min Act: -0.1700

Data Index: 7483771 (Open Web Text)

Max Activating Token Index: 982

Text #2

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.8860. Min Act: -0.1700

Data Index: 7233248 (Open Web Text)

Max Activating Token Index: 670

Text #3

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.8523. Min Act: -0.1700

Data Index: 1299142 (Open Web Text)

Max Activating Token Index: 739

Text #4

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.8537. Min Act: -0.1700

Data Index: 8198687 (Open Web Text)

Max Activating Token Index: 599

Text #5

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.8093. Min Act: -0.1700

Data Index: 2023247 (Open Web Text)

Max Activating Token Index: 667

Text #6

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.8161. Min Act: -0.1700

Data Index: 578492 (Open Web Text)

Max Activating Token Index: 347

Text #7

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.8154. Min Act: -0.1700

Data Index: 657744 (Open Web Text)

Max Activating Token Index: 554

Text #8

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7646. Min Act: -0.1700

Data Index: 7483772 (Open Web Text)

Max Activating Token Index: 49

Text #9

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7669. Min Act: -0.1700

Data Index: 2479501 (Open Web Text)

Max Activating Token Index: 610

Text #10

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7794. Min Act: -0.1700

Data Index: 5765160 (Open Web Text)

Max Activating Token Index: 791

Text #11

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7756. Min Act: -0.1700

Data Index: 8065851 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7800. Min Act: -0.1700

Data Index: 7773332 (Open Web Text)

Max Activating Token Index: 702

Text #13

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7352. Min Act: -0.1700

Data Index: 5455602 (Open Web Text)

Max Activating Token Index: 224

Text #14

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7563. Min Act: -0.1700

Data Index: 2527726 (Open Web Text)

Max Activating Token Index: 872

Text #15

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7683. Min Act: -0.1700

Data Index: 7663292 (Open Web Text)

Max Activating Token Index: 199

Text #16

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7561. Min Act: -0.1700

Data Index: 7546968 (Open Web Text)

Max Activating Token Index: 308

Text #17

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7419. Min Act: -0.1700

Data Index: 118426 (Open Web Text)

Max Activating Token Index: 406

Text #18

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7610. Min Act: -0.1700

Data Index: 156282 (Open Web Text)

Max Activating Token Index: 524

Text #19

Max Range: 5.2347. Min Range: -5.2347

Max Act: 4.7451. Min Act: -0.1700

Data Index: 6047407 (Open Web Text)

Max Activating Token Index: 526