Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1000 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.6168. Min Act: -0.1698

Data Index: 6255363 (Open Web Text)

Max Activating Token Index: 372

Text #1

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.6013. Min Act: -0.1700

Data Index: 8026367 (Open Web Text)

Max Activating Token Index: 114

Text #2

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.6051. Min Act: -0.1595

Data Index: 121125 (Open Web Text)

Max Activating Token Index: 251

Text #3

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5792. Min Act: -0.1650

Data Index: 6822985 (Open Web Text)

Max Activating Token Index: 51

Text #4

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5629. Min Act: -0.1697

Data Index: 3601154 (Open Web Text)

Max Activating Token Index: 412

Text #5

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5657. Min Act: -0.1664

Data Index: 7888363 (Open Web Text)

Max Activating Token Index: 145

Text #6

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5578. Min Act: -0.1659

Data Index: 1173141 (Open Web Text)

Max Activating Token Index: 126

Text #7

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5561. Min Act: -0.1679

Data Index: 900907 (Open Web Text)

Max Activating Token Index: 318

Text #8

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5520. Min Act: -0.1614

Data Index: 5034132 (Open Web Text)

Max Activating Token Index: 97

Text #9

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5416. Min Act: -0.1700

Data Index: 7657253 (Open Web Text)

Max Activating Token Index: 493

Text #10

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5479. Min Act: -0.1646

Data Index: 4715417 (Open Web Text)

Max Activating Token Index: 179

Text #11

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5523. Min Act: -0.1669

Data Index: 7900855 (Open Web Text)

Max Activating Token Index: 327

Text #12

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5427. Min Act: -0.1695

Data Index: 6582387 (Open Web Text)

Max Activating Token Index: 152

Text #13

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5529. Min Act: -0.1700

Data Index: 873568 (Open Web Text)

Max Activating Token Index: 210

Text #14

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5526. Min Act: -0.1657

Data Index: 6580092 (Open Web Text)

Max Activating Token Index: 328

Text #15

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5399. Min Act: -0.1700

Data Index: 2727477 (Open Web Text)

Max Activating Token Index: 198

Text #16

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5398. Min Act: -0.1700

Data Index: 3324809 (Open Web Text)

Max Activating Token Index: 569

Text #17

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5434. Min Act: -0.1694

Data Index: 7485045 (Open Web Text)

Max Activating Token Index: 22

Text #18

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5365. Min Act: -0.1687

Data Index: 5313489 (Open Web Text)

Max Activating Token Index: 94

Text #19

Max Range: 1.6168. Min Range: -1.6168

Max Act: 1.5429. Min Act: -0.1629

Data Index: 5459990 (Open Web Text)

Max Activating Token Index: 374