Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1117 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.9391. Min Act: -0.1700

Data Index: 5697570 (Open Web Text)

Max Activating Token Index: 538

Text #1

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.7618. Min Act: -0.1700

Data Index: 2515321 (Open Web Text)

Max Activating Token Index: 500

Text #2

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.7290. Min Act: -0.1700

Data Index: 1210344 (Open Web Text)

Max Activating Token Index: 171

Text #3

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.7234. Min Act: -0.1700

Data Index: 474767 (Open Web Text)

Max Activating Token Index: 438

Text #4

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6877. Min Act: -0.1700

Data Index: 6330979 (Open Web Text)

Max Activating Token Index: 213

Text #5

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6883. Min Act: -0.1700

Data Index: 1148905 (Open Web Text)

Max Activating Token Index: 376

Text #6

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6854. Min Act: -0.1700

Data Index: 8258895 (Open Web Text)

Max Activating Token Index: 490

Text #7

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6842. Min Act: -0.1700

Data Index: 2917046 (Open Web Text)

Max Activating Token Index: 518

Text #8

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6883. Min Act: -0.0656

Data Index: 8328112 (Open Web Text)

Max Activating Token Index: 822

Text #9

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6831. Min Act: -0.1700

Data Index: 6882797 (Open Web Text)

Max Activating Token Index: 463

Text #10

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6755. Min Act: -0.1700

Data Index: 1148904 (Open Web Text)

Max Activating Token Index: 827

Text #11

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6808. Min Act: -0.1700

Data Index: 2167546 (Open Web Text)

Max Activating Token Index: 488

Text #12

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6784. Min Act: -0.1700

Data Index: 4593390 (Open Web Text)

Max Activating Token Index: 1014

Text #13

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6753. Min Act: -0.1700

Data Index: 474766 (Open Web Text)

Max Activating Token Index: 673

Text #14

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6588. Min Act: -0.1700

Data Index: 2931441 (Open Web Text)

Max Activating Token Index: 91

Text #15

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6678. Min Act: -0.1700

Data Index: 8666799 (Open Web Text)

Max Activating Token Index: 350

Text #16

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6631. Min Act: -0.1700

Data Index: 4454473 (Open Web Text)

Max Activating Token Index: 957

Text #17

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6602. Min Act: -0.1700

Data Index: 140333 (Open Web Text)

Max Activating Token Index: 342

Text #18

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6583. Min Act: -0.1700

Data Index: 2906256 (Open Web Text)

Max Activating Token Index: 712

Text #19

Max Range: 1.9391. Min Range: -1.9391

Max Act: 1.6632. Min Act: -0.1700

Data Index: 7620094 (Open Web Text)

Max Activating Token Index: 684