Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2683 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.5228. Min Act: -0.1700

Data Index: 7267614 (Open Web Text)

Max Activating Token Index: 553

Text #1

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.3511. Min Act: -0.1700

Data Index: 2819945 (Open Web Text)

Max Activating Token Index: 347

Text #2

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.3500. Min Act: -0.1700

Data Index: 4159363 (Open Web Text)

Max Activating Token Index: 482

Text #3

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.2391. Min Act: -0.1700

Data Index: 4829057 (Open Web Text)

Max Activating Token Index: 354

Text #4

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1835. Min Act: -0.1700

Data Index: 18562 (Open Web Text)

Max Activating Token Index: 701

Text #5

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.2203. Min Act: -0.1700

Data Index: 5551236 (Open Web Text)

Max Activating Token Index: 899

Text #6

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1526. Min Act: -0.1700

Data Index: 8496624 (Open Web Text)

Max Activating Token Index: 538

Text #7

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1629. Min Act: -0.1700

Data Index: 1394491 (Open Web Text)

Max Activating Token Index: 663

Text #8

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1430. Min Act: -0.1700

Data Index: 7203149 (Open Web Text)

Max Activating Token Index: 970

Text #9

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.2111. Min Act: -0.1700

Data Index: 5926772 (Open Web Text)

Max Activating Token Index: 559

Text #10

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1328. Min Act: -0.1700

Data Index: 7212388 (Open Web Text)

Max Activating Token Index: 209

Text #11

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1502. Min Act: -0.1700

Data Index: 8096129 (Open Web Text)

Max Activating Token Index: 768

Text #12

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1216. Min Act: -0.1700

Data Index: 2201118 (Open Web Text)

Max Activating Token Index: 793

Text #13

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1455. Min Act: -0.1700

Data Index: 1565788 (Open Web Text)

Max Activating Token Index: 466

Text #14

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1655. Min Act: -0.1700

Data Index: 4992192 (Open Web Text)

Max Activating Token Index: 975

Text #15

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1536. Min Act: -0.1700

Data Index: 1840869 (Open Web Text)

Max Activating Token Index: 671

Text #16

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1192. Min Act: -0.1700

Data Index: 2128023 (Open Web Text)

Max Activating Token Index: 858

Text #17

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1278. Min Act: -0.1700

Data Index: 8044658 (Open Web Text)

Max Activating Token Index: 522

Text #18

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1342. Min Act: -0.1700

Data Index: 3645450 (Open Web Text)

Max Activating Token Index: 538

Text #19

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.1126. Min Act: -0.1700

Data Index: 300815 (Open Web Text)

Max Activating Token Index: 207