Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1849 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.3338. Min Act: -0.1700

Data Index: 4022071 (Open Web Text)

Max Activating Token Index: 545

Text #1

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.2838. Min Act: -0.1700

Data Index: 63542 (Open Web Text)

Max Activating Token Index: 596

Text #2

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.2902. Min Act: -0.1700

Data Index: 3337185 (Open Web Text)

Max Activating Token Index: 772

Text #3

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.3042. Min Act: -0.1700

Data Index: 6329810 (Open Web Text)

Max Activating Token Index: 621

Text #4

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1420. Min Act: -0.1700

Data Index: 4491360 (Open Web Text)

Max Activating Token Index: 310

Text #5

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1390. Min Act: -0.1700

Data Index: 8023025 (Open Web Text)

Max Activating Token Index: 633

Text #6

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1347. Min Act: -0.1700

Data Index: 3762863 (Open Web Text)

Max Activating Token Index: 402

Text #7

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1062. Min Act: -0.1700

Data Index: 3250470 (Open Web Text)

Max Activating Token Index: 655

Text #8

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1200. Min Act: -0.1700

Data Index: 5336767 (Open Web Text)

Max Activating Token Index: 956

Text #9

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1112. Min Act: -0.1700

Data Index: 3181004 (Open Web Text)

Max Activating Token Index: 536

Text #10

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1102. Min Act: -0.1700

Data Index: 5218100 (Open Web Text)

Max Activating Token Index: 97

Text #11

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1345. Min Act: -0.1700

Data Index: 5191998 (Open Web Text)

Max Activating Token Index: 157

Text #12

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.0590. Min Act: -0.1700

Data Index: 731556 (Open Web Text)

Max Activating Token Index: 546

Text #13

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.0627. Min Act: -0.1700

Data Index: 6500441 (Open Web Text)

Max Activating Token Index: 789

Text #14

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.0486. Min Act: -0.1700

Data Index: 4390556 (Open Web Text)

Max Activating Token Index: 896

Text #15

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.1002. Min Act: -0.1700

Data Index: 2645863 (Open Web Text)

Max Activating Token Index: 122

Text #16

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.0434. Min Act: -0.1700

Data Index: 4745570 (Open Web Text)

Max Activating Token Index: 400

Text #17

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.0680. Min Act: -0.1700

Data Index: 2212457 (Open Web Text)

Max Activating Token Index: 389

Text #18

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.0324. Min Act: -0.1700

Data Index: 6310504 (Open Web Text)

Max Activating Token Index: 303

Text #19

Max Range: 4.3338. Min Range: -4.3338

Max Act: 4.0465. Min Act: -0.1700

Data Index: 5222907 (Open Web Text)

Max Activating Token Index: 252