Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 5 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.6873. Min Act: -0.1700

Data Index: 7075907 (Open Web Text)

Max Activating Token Index: 850

Text #1

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.2563. Min Act: -0.1700

Data Index: 4160165 (Open Web Text)

Max Activating Token Index: 99

Text #2

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.2389. Min Act: -0.1700

Data Index: 7346895 (Open Web Text)

Max Activating Token Index: 652

Text #3

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.2161. Min Act: -0.1700

Data Index: 4780217 (Open Web Text)

Max Activating Token Index: 1018

Text #4

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1784. Min Act: -0.1700

Data Index: 1195034 (Open Web Text)

Max Activating Token Index: 805

Text #5

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1872. Min Act: -0.1700

Data Index: 4734696 (Open Web Text)

Max Activating Token Index: 681

Text #6

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1577. Min Act: -0.1700

Data Index: 8311132 (Open Web Text)

Max Activating Token Index: 533

Text #7

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1720. Min Act: -0.1700

Data Index: 2202415 (Open Web Text)

Max Activating Token Index: 336

Text #8

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1794. Min Act: -0.1700

Data Index: 3269637 (Open Web Text)

Max Activating Token Index: 695

Text #9

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1586. Min Act: -0.1700

Data Index: 1992665 (Open Web Text)

Max Activating Token Index: 392

Text #10

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1463. Min Act: -0.1700

Data Index: 4164786 (Open Web Text)

Max Activating Token Index: 995

Text #11

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1708. Min Act: -0.1700

Data Index: 8140264 (Open Web Text)

Max Activating Token Index: 163

Text #12

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1438. Min Act: -0.1700

Data Index: 1330647 (Open Web Text)

Max Activating Token Index: 833

Text #13

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1444. Min Act: -0.1700

Data Index: 5916609 (Open Web Text)

Max Activating Token Index: 599

Text #14

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1402. Min Act: -0.1700

Data Index: 1852656 (Open Web Text)

Max Activating Token Index: 847

Text #15

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1462. Min Act: -0.1700

Data Index: 4345816 (Open Web Text)

Max Activating Token Index: 850

Text #16

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1516. Min Act: -0.1700

Data Index: 5352024 (Open Web Text)

Max Activating Token Index: 273

Text #17

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1368. Min Act: -0.1700

Data Index: 1221821 (Open Web Text)

Max Activating Token Index: 178

Text #18

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1235. Min Act: -0.1700

Data Index: 4485357 (Open Web Text)

Max Activating Token Index: 497

Text #19

Max Range: 3.6873. Min Range: -3.6873

Max Act: 3.1323. Min Act: -0.1700

Data Index: 2471204 (Open Web Text)

Max Activating Token Index: 664