Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2473 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.4806. Min Act: -0.1700

Data Index: 7903578 (Open Web Text)

Max Activating Token Index: 24

Text #1

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.4002. Min Act: -0.1700

Data Index: 5652622 (Open Web Text)

Max Activating Token Index: 972

Text #2

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3547. Min Act: -0.1700

Data Index: 1968720 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3152. Min Act: -0.1700

Data Index: 98427 (Open Web Text)

Max Activating Token Index: 515

Text #4

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3403. Min Act: -0.1700

Data Index: 7105513 (Open Web Text)

Max Activating Token Index: 8

Text #5

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3459. Min Act: -0.1700

Data Index: 6997610 (Open Web Text)

Max Activating Token Index: 259

Text #6

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3291. Min Act: -0.1700

Data Index: 3885130 (Open Web Text)

Max Activating Token Index: 220

Text #7

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2904. Min Act: -0.1700

Data Index: 1420135 (Open Web Text)

Max Activating Token Index: 848

Text #8

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3060. Min Act: -0.1700

Data Index: 4443989 (Open Web Text)

Max Activating Token Index: 319

Text #9

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3033. Min Act: -0.1700

Data Index: 8242361 (Open Web Text)

Max Activating Token Index: 277

Text #10

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.3030. Min Act: -0.1700

Data Index: 114873 (Open Web Text)

Max Activating Token Index: 35

Text #11

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2957. Min Act: -0.1700

Data Index: 2466726 (Open Web Text)

Max Activating Token Index: 927

Text #12

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2609. Min Act: -0.1700

Data Index: 7318487 (Open Web Text)

Max Activating Token Index: 16

Text #13

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2999. Min Act: -0.1700

Data Index: 8197213 (Open Web Text)

Max Activating Token Index: 885

Text #14

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2925. Min Act: -0.1700

Data Index: 246750 (Open Web Text)

Max Activating Token Index: 315

Text #15

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2572. Min Act: -0.1700

Data Index: 6924587 (Open Web Text)

Max Activating Token Index: 1012

Text #16

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2361. Min Act: -0.1700

Data Index: 4407925 (Open Web Text)

Max Activating Token Index: 292

Text #17

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2523. Min Act: -0.1700

Data Index: 7365633 (Open Web Text)

Max Activating Token Index: 864

Text #18

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2477. Min Act: -0.1700

Data Index: 2064092 (Open Web Text)

Max Activating Token Index: 198

Text #19

Max Range: 4.4806. Min Range: -4.4806

Max Act: 4.2463. Min Act: -0.1700

Data Index: 3463549 (Open Web Text)

Max Activating Token Index: 165