Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 213 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.3294. Min Act: -0.1700

Data Index: 274289 (Open Web Text)

Max Activating Token Index: 143

Text #1

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.2971. Min Act: -0.1700

Data Index: 7655908 (Open Web Text)

Max Activating Token Index: 274

Text #2

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.2778. Min Act: -0.1700

Data Index: 5602503 (Open Web Text)

Max Activating Token Index: 62

Text #3

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.2612. Min Act: -0.1700

Data Index: 8159659 (Open Web Text)

Max Activating Token Index: 56

Text #4

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.2470. Min Act: -0.1700

Data Index: 8244201 (Open Web Text)

Max Activating Token Index: 275

Text #5

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.2167. Min Act: -0.1700

Data Index: 8729437 (Open Web Text)

Max Activating Token Index: 915

Text #6

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.2186. Min Act: -0.1700

Data Index: 909210 (Open Web Text)

Max Activating Token Index: 619

Text #7

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.2254. Min Act: -0.1700

Data Index: 2189288 (Open Web Text)

Max Activating Token Index: 659

Text #8

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1904. Min Act: -0.1700

Data Index: 969903 (Open Web Text)

Max Activating Token Index: 872

Text #9

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1706. Min Act: -0.1700

Data Index: 5541727 (Open Web Text)

Max Activating Token Index: 575

Text #10

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1649. Min Act: -0.1700

Data Index: 4220474 (Open Web Text)

Max Activating Token Index: 254

Text #11

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1704. Min Act: -0.1700

Data Index: 1048979 (Open Web Text)

Max Activating Token Index: 538

Text #12

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1542. Min Act: -0.1700

Data Index: 7913207 (Open Web Text)

Max Activating Token Index: 570

Text #13

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1501. Min Act: -0.1700

Data Index: 3959894 (Open Web Text)

Max Activating Token Index: 183

Text #14

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1653. Min Act: -0.1700

Data Index: 8303906 (Open Web Text)

Max Activating Token Index: 795

Text #15

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1294. Min Act: -0.1700

Data Index: 1485282 (Open Web Text)

Max Activating Token Index: 564

Text #16

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1377. Min Act: -0.1700

Data Index: 3067050 (Open Web Text)

Max Activating Token Index: 445

Text #17

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1256. Min Act: -0.1700

Data Index: 3867962 (Open Web Text)

Max Activating Token Index: 674

Text #18

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1335. Min Act: -0.1700

Data Index: 7544815 (Open Web Text)

Max Activating Token Index: 733

Text #19

Max Range: 3.3294. Min Range: -3.3294

Max Act: 3.1317. Min Act: -0.1700

Data Index: 2615501 (Open Web Text)

Max Activating Token Index: 108