Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 187 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.3028. Min Act: -0.1700

Data Index: 2083733 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.2531. Min Act: -0.1700

Data Index: 2605560 (Open Web Text)

Max Activating Token Index: 989

Text #2

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.1863. Min Act: -0.1700

Data Index: 8182476 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.1663. Min Act: -0.1700

Data Index: 5165319 (Open Web Text)

Max Activating Token Index: 23

Text #4

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.1392. Min Act: -0.1700

Data Index: 997638 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.1603. Min Act: -0.1700

Data Index: 4499458 (Open Web Text)

Max Activating Token Index: 284

Text #6

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.1617. Min Act: -0.1700

Data Index: 8628176 (Open Web Text)

Max Activating Token Index: 838

Text #7

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.1392. Min Act: -0.1700

Data Index: 359199 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.1090. Min Act: -0.1700

Data Index: 5998807 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0854. Min Act: -0.1700

Data Index: 4156275 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0741. Min Act: -0.1700

Data Index: 5419128 (Open Web Text)

Max Activating Token Index: 60

Text #11

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0980. Min Act: -0.1700

Data Index: 1742845 (Open Web Text)

Max Activating Token Index: 427

Text #12

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0912. Min Act: -0.1700

Data Index: 7215582 (Open Web Text)

Max Activating Token Index: 769

Text #13

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0968. Min Act: -0.1700

Data Index: 8597013 (Open Web Text)

Max Activating Token Index: 976

Text #14

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0595. Min Act: -0.1700

Data Index: 1480132 (Open Web Text)

Max Activating Token Index: 355

Text #15

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0568. Min Act: -0.1700

Data Index: 3508591 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0592. Min Act: -0.1700

Data Index: 3541221 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0582. Min Act: -0.1700

Data Index: 3629412 (Open Web Text)

Max Activating Token Index: 830

Text #18

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0432. Min Act: -0.1700

Data Index: 8019852 (Open Web Text)

Max Activating Token Index: 14

Text #19

Max Range: 4.3028. Min Range: -4.3028

Max Act: 4.0629. Min Act: -0.1700

Data Index: 5269204 (Open Web Text)

Max Activating Token Index: 685