Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1280 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1323. Min Range: -5.1323

Max Act: 5.1323. Min Act: -0.1700

Data Index: 2698739 (Open Web Text)

Max Activating Token Index: 284

Text #1

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.9053. Min Act: -0.1700

Data Index: 7899774 (Open Web Text)

Max Activating Token Index: 295

Text #2

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.8709. Min Act: -0.1700

Data Index: 6691253 (Open Web Text)

Max Activating Token Index: 765

Text #3

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7990. Min Act: -0.1700

Data Index: 5866480 (Open Web Text)

Max Activating Token Index: 378

Text #4

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7827. Min Act: -0.1700

Data Index: 5570605 (Open Web Text)

Max Activating Token Index: 533

Text #5

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7547. Min Act: -0.1700

Data Index: 7937374 (Open Web Text)

Max Activating Token Index: 738

Text #6

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7871. Min Act: -0.1700

Data Index: 282801 (Open Web Text)

Max Activating Token Index: 514

Text #7

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7702. Min Act: -0.1700

Data Index: 1826576 (Open Web Text)

Max Activating Token Index: 851

Text #8

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7716. Min Act: -0.1700

Data Index: 2108824 (Open Web Text)

Max Activating Token Index: 503

Text #9

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7652. Min Act: -0.1700

Data Index: 8231635 (Open Web Text)

Max Activating Token Index: 64

Text #10

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7414. Min Act: -0.1700

Data Index: 3275819 (Open Web Text)

Max Activating Token Index: 279

Text #11

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.7075. Min Act: -0.1700

Data Index: 2065637 (Open Web Text)

Max Activating Token Index: 854

Text #12

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6849. Min Act: -0.1700

Data Index: 763009 (Open Web Text)

Max Activating Token Index: 245

Text #13

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6414. Min Act: -0.1700

Data Index: 6756769 (Open Web Text)

Max Activating Token Index: 879

Text #14

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6658. Min Act: -0.1700

Data Index: 6837854 (Open Web Text)

Max Activating Token Index: 1013

Text #15

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6518. Min Act: -0.1700

Data Index: 5785731 (Open Web Text)

Max Activating Token Index: 200

Text #16

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6733. Min Act: -0.1700

Data Index: 5273032 (Open Web Text)

Max Activating Token Index: 257

Text #17

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6440. Min Act: -0.1700

Data Index: 3200572 (Open Web Text)

Max Activating Token Index: 310

Text #18

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6782. Min Act: -0.1700

Data Index: 3388530 (Open Web Text)

Max Activating Token Index: 872

Text #19

Max Range: 5.1323. Min Range: -5.1323

Max Act: 4.6550. Min Act: -0.1700

Data Index: 1123091 (Open Web Text)

Max Activating Token Index: 715