Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3043 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.8980. Min Act: -0.1700

Data Index: 7991006 (Open Web Text)

Max Activating Token Index: 558

Text #1

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.8626. Min Act: -0.1700

Data Index: 5263812 (Open Web Text)

Max Activating Token Index: 650

Text #2

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.8488. Min Act: -0.1700

Data Index: 8208557 (Open Web Text)

Max Activating Token Index: 359

Text #3

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.8366. Min Act: -0.1700

Data Index: 5441708 (Open Web Text)

Max Activating Token Index: 667

Text #4

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7899. Min Act: -0.1700

Data Index: 339908 (Open Web Text)

Max Activating Token Index: 412

Text #5

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.8074. Min Act: -0.1700

Data Index: 5441707 (Open Web Text)

Max Activating Token Index: 994

Text #6

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7779. Min Act: -0.1700

Data Index: 368765 (Open Web Text)

Max Activating Token Index: 510

Text #7

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7450. Min Act: -0.1700

Data Index: 7511499 (Open Web Text)

Max Activating Token Index: 345

Text #8

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7248. Min Act: -0.1700

Data Index: 6384859 (Open Web Text)

Max Activating Token Index: 261

Text #9

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.6981. Min Act: -0.1700

Data Index: 4770065 (Open Web Text)

Max Activating Token Index: 671

Text #10

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7181. Min Act: -0.1700

Data Index: 102773 (Open Web Text)

Max Activating Token Index: 997

Text #11

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7145. Min Act: -0.1700

Data Index: 1228300 (Open Web Text)

Max Activating Token Index: 472

Text #12

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7208. Min Act: -0.1700

Data Index: 2924553 (Open Web Text)

Max Activating Token Index: 1006

Text #13

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.6869. Min Act: -0.1700

Data Index: 975950 (Open Web Text)

Max Activating Token Index: 420

Text #14

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.6714. Min Act: -0.1700

Data Index: 4515582 (Open Web Text)

Max Activating Token Index: 612

Text #15

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.6962. Min Act: -0.1700

Data Index: 6411668 (Open Web Text)

Max Activating Token Index: 994

Text #16

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.6762. Min Act: -0.1700

Data Index: 4406325 (Open Web Text)

Max Activating Token Index: 107

Text #17

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.6848. Min Act: -0.1700

Data Index: 2142283 (Open Web Text)

Max Activating Token Index: 838

Text #18

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.6922. Min Act: -0.1700

Data Index: 4382824 (Open Web Text)

Max Activating Token Index: 369

Text #19

Max Range: 5.8980. Min Range: -5.8980

Max Act: 5.7020. Min Act: -0.1700

Data Index: 1755948 (Open Web Text)

Max Activating Token Index: 966