Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2554 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.7307. Min Act: -0.1698

Data Index: 8061751 (Open Web Text)

Max Activating Token Index: 300

Text #1

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.7354. Min Act: -0.1700

Data Index: 3840844 (Open Web Text)

Max Activating Token Index: 446

Text #2

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.7055. Min Act: -0.1700

Data Index: 2347268 (Open Web Text)

Max Activating Token Index: 658

Text #3

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.7008. Min Act: -0.1700

Data Index: 3484859 (Open Web Text)

Max Activating Token Index: 491

Text #4

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6869. Min Act: -0.1700

Data Index: 2831458 (Open Web Text)

Max Activating Token Index: 162

Text #5

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6842. Min Act: -0.1700

Data Index: 6329940 (Open Web Text)

Max Activating Token Index: 279

Text #6

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6707. Min Act: -0.1700

Data Index: 1170414 (Open Web Text)

Max Activating Token Index: 457

Text #7

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6788. Min Act: -0.1700

Data Index: 6204399 (Open Web Text)

Max Activating Token Index: 432

Text #8

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6697. Min Act: -0.1700

Data Index: 1656115 (Open Web Text)

Max Activating Token Index: 424

Text #9

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6808. Min Act: -0.1700

Data Index: 5359832 (Open Web Text)

Max Activating Token Index: 118

Text #10

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6728. Min Act: -0.1698

Data Index: 6181202 (Open Web Text)

Max Activating Token Index: 355

Text #11

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6636. Min Act: -0.1700

Data Index: 2653322 (Open Web Text)

Max Activating Token Index: 561

Text #12

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6594. Min Act: -0.1700

Data Index: 1709925 (Open Web Text)

Max Activating Token Index: 96

Text #13

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6509. Min Act: -0.1700

Data Index: 1230552 (Open Web Text)

Max Activating Token Index: 541

Text #14

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6553. Min Act: -0.1700

Data Index: 8551789 (Open Web Text)

Max Activating Token Index: 741

Text #15

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6558. Min Act: -0.1700

Data Index: 3407943 (Open Web Text)

Max Activating Token Index: 199

Text #16

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6562. Min Act: -0.1700

Data Index: 5672573 (Open Web Text)

Max Activating Token Index: 238

Text #17

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6662. Min Act: -0.1700

Data Index: 7626576 (Open Web Text)

Max Activating Token Index: 349

Text #18

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6498. Min Act: -0.1692

Data Index: 7190114 (Open Web Text)

Max Activating Token Index: 856

Text #19

Max Range: 1.7354. Min Range: -1.7354

Max Act: 1.6418. Min Act: -0.1700

Data Index: 95870 (Open Web Text)

Max Activating Token Index: 467