Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 932 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8838. Min Act: -0.1700

Data Index: 8782985 (Open Web Text)

Max Activating Token Index: 416

Text #1

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7670. Min Act: -0.1700

Data Index: 7723857 (Open Web Text)

Max Activating Token Index: 55

Text #2

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7416. Min Act: -0.1700

Data Index: 7965656 (Open Web Text)

Max Activating Token Index: 187

Text #3

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7255. Min Act: -0.1700

Data Index: 7612716 (Open Web Text)

Max Activating Token Index: 898

Text #4

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7079. Min Act: -0.1700

Data Index: 195290 (Open Web Text)

Max Activating Token Index: 71

Text #5

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7148. Min Act: -0.1700

Data Index: 7623899 (Open Web Text)

Max Activating Token Index: 511

Text #6

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7043. Min Act: -0.1700

Data Index: 6749912 (Open Web Text)

Max Activating Token Index: 582

Text #7

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6793. Min Act: -0.1700

Data Index: 1647826 (Open Web Text)

Max Activating Token Index: 763

Text #8

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6789. Min Act: -0.1700

Data Index: 4881339 (Open Web Text)

Max Activating Token Index: 240

Text #9

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6683. Min Act: -0.1700

Data Index: 1157453 (Open Web Text)

Max Activating Token Index: 321

Text #10

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6681. Min Act: -0.1700

Data Index: 3444344 (Open Web Text)

Max Activating Token Index: 985

Text #11

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6677. Min Act: -0.1700

Data Index: 899619 (Open Web Text)

Max Activating Token Index: 349

Text #12

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6526. Min Act: -0.1700

Data Index: 3124939 (Open Web Text)

Max Activating Token Index: 419

Text #13

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6423. Min Act: -0.1700

Data Index: 7806117 (Open Web Text)

Max Activating Token Index: 783

Text #14

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6513. Min Act: -0.1700

Data Index: 3401252 (Open Web Text)

Max Activating Token Index: 354

Text #15

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6659. Min Act: -0.1700

Data Index: 4625510 (Open Web Text)

Max Activating Token Index: 536

Text #16

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6454. Min Act: -0.1700

Data Index: 1820527 (Open Web Text)

Max Activating Token Index: 374

Text #17

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6129. Min Act: -0.1700

Data Index: 8490218 (Open Web Text)

Max Activating Token Index: 649

Text #18

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6389. Min Act: -0.1700

Data Index: 1424434 (Open Web Text)

Max Activating Token Index: 271

Text #19

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6453. Min Act: -0.1700

Data Index: 1610161 (Open Web Text)

Max Activating Token Index: 697