Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 795 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.7582. Min Act: -0.1700

Data Index: 2387080 (Open Web Text)

Max Activating Token Index: 548

Text #1

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.7644. Min Act: -0.1700

Data Index: 21329 (Open Web Text)

Max Activating Token Index: 527

Text #2

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.7127. Min Act: -0.1700

Data Index: 5257434 (Open Web Text)

Max Activating Token Index: 148

Text #3

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6762. Min Act: -0.1700

Data Index: 6424863 (Open Web Text)

Max Activating Token Index: 667

Text #4

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6960. Min Act: -0.1700

Data Index: 3990182 (Open Web Text)

Max Activating Token Index: 379

Text #5

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6825. Min Act: -0.1700

Data Index: 4204139 (Open Web Text)

Max Activating Token Index: 560

Text #6

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6975. Min Act: -0.1700

Data Index: 587538 (Open Web Text)

Max Activating Token Index: 750

Text #7

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.7222. Min Act: -0.1700

Data Index: 4972914 (Open Web Text)

Max Activating Token Index: 355

Text #8

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.7224. Min Act: -0.1700

Data Index: 4721073 (Open Web Text)

Max Activating Token Index: 568

Text #9

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.7305. Min Act: -0.1700

Data Index: 280232 (Open Web Text)

Max Activating Token Index: 523

Text #10

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6634. Min Act: -0.1700

Data Index: 3361168 (Open Web Text)

Max Activating Token Index: 288

Text #11

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6539. Min Act: -0.1700

Data Index: 6266967 (Open Web Text)

Max Activating Token Index: 910

Text #12

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6936. Min Act: -0.1700

Data Index: 6075263 (Open Web Text)

Max Activating Token Index: 330

Text #13

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6410. Min Act: -0.1700

Data Index: 3295602 (Open Web Text)

Max Activating Token Index: 466

Text #14

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6705. Min Act: -0.1700

Data Index: 4635842 (Open Web Text)

Max Activating Token Index: 905

Text #15

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6513. Min Act: -0.1700

Data Index: 7688388 (Open Web Text)

Max Activating Token Index: 626

Text #16

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6399. Min Act: -0.1700

Data Index: 7545058 (Open Web Text)

Max Activating Token Index: 483

Text #17

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6149. Min Act: -0.1700

Data Index: 2614675 (Open Web Text)

Max Activating Token Index: 768

Text #18

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6548. Min Act: -0.1700

Data Index: 6902435 (Open Web Text)

Max Activating Token Index: 675

Text #19

Max Range: 3.7644. Min Range: -3.7644

Max Act: 3.6349. Min Act: -0.1700

Data Index: 3775206 (Open Web Text)

Max Activating Token Index: 706