Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1387 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.5951. Min Act: -0.1700

Data Index: 1467610 (Open Web Text)

Max Activating Token Index: 310

Text #1

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.5311. Min Act: -0.1700

Data Index: 2135533 (Open Web Text)

Max Activating Token Index: 450

Text #2

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4653. Min Act: -0.1700

Data Index: 8616190 (Open Web Text)

Max Activating Token Index: 300

Text #3

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4815. Min Act: -0.1700

Data Index: 5730889 (Open Web Text)

Max Activating Token Index: 605

Text #4

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4578. Min Act: -0.1700

Data Index: 204541 (Open Web Text)

Max Activating Token Index: 961

Text #5

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4437. Min Act: -0.1700

Data Index: 7142782 (Open Web Text)

Max Activating Token Index: 884

Text #6

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.3794. Min Act: -0.1700

Data Index: 1561882 (Open Web Text)

Max Activating Token Index: 1007

Text #7

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.3380. Min Act: -0.1700

Data Index: 4842970 (Open Web Text)

Max Activating Token Index: 531

Text #8

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.2663. Min Act: -0.1700

Data Index: 876128 (Open Web Text)

Max Activating Token Index: 757

Text #9

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.2873. Min Act: -0.1700

Data Index: 1424558 (Open Web Text)

Max Activating Token Index: 542

Text #10

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.2347. Min Act: -0.1700

Data Index: 8102600 (Open Web Text)

Max Activating Token Index: 760

Text #11

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.2316. Min Act: -0.1700

Data Index: 241097 (Open Web Text)

Max Activating Token Index: 898

Text #12

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.2103. Min Act: -0.1700

Data Index: 2484237 (Open Web Text)

Max Activating Token Index: 687

Text #13

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.2462. Min Act: -0.1700

Data Index: 2843665 (Open Web Text)

Max Activating Token Index: 512

Text #14

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.1930. Min Act: -0.1700

Data Index: 1279518 (Open Web Text)

Max Activating Token Index: 242

Text #15

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.1954. Min Act: -0.1700

Data Index: 5570459 (Open Web Text)

Max Activating Token Index: 742

Text #16

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.1926. Min Act: -0.1700

Data Index: 8264452 (Open Web Text)

Max Activating Token Index: 495

Text #17

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.2423. Min Act: -0.1700

Data Index: 1612996 (Open Web Text)

Max Activating Token Index: 304

Text #18

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.1899. Min Act: -0.1700

Data Index: 4616342 (Open Web Text)

Max Activating Token Index: 886

Text #19

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.1590. Min Act: -0.1700

Data Index: 7945282 (Open Web Text)

Max Activating Token Index: 219