Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 613 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.7278. Min Act: -0.1700

Data Index: 5621684 (Open Web Text)

Max Activating Token Index: 589

Text #1

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.5395. Min Act: -0.1700

Data Index: 5991881 (Open Web Text)

Max Activating Token Index: 884

Text #2

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4514. Min Act: -0.1700

Data Index: 5966741 (Open Web Text)

Max Activating Token Index: 440

Text #3

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.4422. Min Act: -0.1700

Data Index: 2449139 (Open Web Text)

Max Activating Token Index: 411

Text #4

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.3861. Min Act: -0.1700

Data Index: 7674172 (Open Web Text)

Max Activating Token Index: 761

Text #5

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.2829. Min Act: -0.1700

Data Index: 3151110 (Open Web Text)

Max Activating Token Index: 617

Text #6

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.2282. Min Act: -0.1700

Data Index: 2419136 (Open Web Text)

Max Activating Token Index: 752

Text #7

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.2329. Min Act: -0.1700

Data Index: 74529 (Open Web Text)

Max Activating Token Index: 937

Text #8

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.2188. Min Act: -0.1700

Data Index: 6125964 (Open Web Text)

Max Activating Token Index: 432

Text #9

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.2326. Min Act: -0.1700

Data Index: 7162764 (Open Web Text)

Max Activating Token Index: 961

Text #10

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.2606. Min Act: -0.1700

Data Index: 2110912 (Open Web Text)

Max Activating Token Index: 782

Text #11

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1345. Min Act: -0.1700

Data Index: 6455168 (Open Web Text)

Max Activating Token Index: 639

Text #12

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1560. Min Act: -0.1700

Data Index: 5764218 (Open Web Text)

Max Activating Token Index: 776

Text #13

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1530. Min Act: -0.1700

Data Index: 6354136 (Open Web Text)

Max Activating Token Index: 909

Text #14

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1709. Min Act: -0.1700

Data Index: 3519671 (Open Web Text)

Max Activating Token Index: 155

Text #15

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1771. Min Act: -0.1700

Data Index: 1353894 (Open Web Text)

Max Activating Token Index: 297

Text #16

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1776. Min Act: -0.1700

Data Index: 4887007 (Open Web Text)

Max Activating Token Index: 319

Text #17

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1197. Min Act: -0.1700

Data Index: 2721213 (Open Web Text)

Max Activating Token Index: 563

Text #18

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1314. Min Act: -0.1700

Data Index: 4740090 (Open Web Text)

Max Activating Token Index: 753

Text #19

Max Range: 3.7278. Min Range: -3.7278

Max Act: 3.1198. Min Act: -0.1700

Data Index: 1711829 (Open Web Text)

Max Activating Token Index: 64