Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 365 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.3568. Min Act: -0.1700

Data Index: 8635074 (Open Web Text)

Max Activating Token Index: 436

Text #1

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.1646. Min Act: -0.1700

Data Index: 3404892 (Open Web Text)

Max Activating Token Index: 488

Text #2

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.1131. Min Act: -0.1700

Data Index: 5448573 (Open Web Text)

Max Activating Token Index: 405

Text #3

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.1188. Min Act: -0.1700

Data Index: 2882190 (Open Web Text)

Max Activating Token Index: 186

Text #4

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0741. Min Act: -0.1700

Data Index: 5086699 (Open Web Text)

Max Activating Token Index: 236

Text #5

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0784. Min Act: -0.1700

Data Index: 1599253 (Open Web Text)

Max Activating Token Index: 281

Text #6

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0960. Min Act: -0.1700

Data Index: 270192 (Open Web Text)

Max Activating Token Index: 996

Text #7

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0721. Min Act: -0.1700

Data Index: 1138522 (Open Web Text)

Max Activating Token Index: 399

Text #8

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.1060. Min Act: -0.1700

Data Index: 658161 (Open Web Text)

Max Activating Token Index: 363

Text #9

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0821. Min Act: -0.1700

Data Index: 1806473 (Open Web Text)

Max Activating Token Index: 465

Text #10

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0830. Min Act: -0.1700

Data Index: 8585313 (Open Web Text)

Max Activating Token Index: 218

Text #11

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.1132. Min Act: -0.1700

Data Index: 3957357 (Open Web Text)

Max Activating Token Index: 676

Text #12

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.1071. Min Act: -0.1700

Data Index: 7244100 (Open Web Text)

Max Activating Token Index: 663

Text #13

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0693. Min Act: -0.1700

Data Index: 3538544 (Open Web Text)

Max Activating Token Index: 976

Text #14

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0492. Min Act: -0.1700

Data Index: 4508279 (Open Web Text)

Max Activating Token Index: 593

Text #15

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0655. Min Act: -0.1700

Data Index: 4404918 (Open Web Text)

Max Activating Token Index: 106

Text #16

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0770. Min Act: -0.1700

Data Index: 7941797 (Open Web Text)

Max Activating Token Index: 11

Text #17

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0928. Min Act: -0.1700

Data Index: 6113564 (Open Web Text)

Max Activating Token Index: 301

Text #18

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0329. Min Act: -0.1700

Data Index: 3292485 (Open Web Text)

Max Activating Token Index: 25

Text #19

Max Range: 4.3568. Min Range: -4.3568

Max Act: 4.0348. Min Act: -0.1700

Data Index: 1208274 (Open Web Text)

Max Activating Token Index: 270