Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2170 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.3463. Min Act: -0.1700

Data Index: 5628747 (Open Web Text)

Max Activating Token Index: 858

Text #1

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.3138. Min Act: -0.1700

Data Index: 5278956 (Open Web Text)

Max Activating Token Index: 757

Text #2

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.2484. Min Act: -0.1700

Data Index: 6396223 (Open Web Text)

Max Activating Token Index: 636

Text #3

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.2134. Min Act: -0.1700

Data Index: 7776976 (Open Web Text)

Max Activating Token Index: 106

Text #4

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.2121. Min Act: -0.1700

Data Index: 7812861 (Open Web Text)

Max Activating Token Index: 462

Text #5

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.1997. Min Act: -0.1700

Data Index: 7692607 (Open Web Text)

Max Activating Token Index: 701

Text #6

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.1619. Min Act: -0.1700

Data Index: 3853095 (Open Web Text)

Max Activating Token Index: 606

Text #7

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.1439. Min Act: -0.1700

Data Index: 5628746 (Open Web Text)

Max Activating Token Index: 265

Text #8

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.0881. Min Act: -0.1700

Data Index: 8271707 (Open Web Text)

Max Activating Token Index: 356

Text #9

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.1208. Min Act: -0.1700

Data Index: 1462277 (Open Web Text)

Max Activating Token Index: 378

Text #10

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.1068. Min Act: -0.1700

Data Index: 419113 (Open Web Text)

Max Activating Token Index: 555

Text #11

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.1076. Min Act: -0.1700

Data Index: 4250784 (Open Web Text)

Max Activating Token Index: 649

Text #12

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.0737. Min Act: -0.1700

Data Index: 2639930 (Open Web Text)

Max Activating Token Index: 628

Text #13

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.0161. Min Act: -0.1700

Data Index: 4460064 (Open Web Text)

Max Activating Token Index: 258

Text #14

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.0532. Min Act: -0.1700

Data Index: 7689747 (Open Web Text)

Max Activating Token Index: 146

Text #15

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.0368. Min Act: -0.1700

Data Index: 1015292 (Open Web Text)

Max Activating Token Index: 347

Text #16

Max Range: 4.3463. Min Range: -4.3463

Max Act: 4.0177. Min Act: -0.1700

Data Index: 362947 (Open Web Text)

Max Activating Token Index: 841

Text #17

Max Range: 4.3463. Min Range: -4.3463

Max Act: 3.9864. Min Act: -0.1700

Data Index: 70538 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 4.3463. Min Range: -4.3463

Max Act: 3.9833. Min Act: -0.1700

Data Index: 2949744 (Open Web Text)

Max Activating Token Index: 591

Text #19

Max Range: 4.3463. Min Range: -4.3463

Max Act: 3.9875. Min Act: -0.1700

Data Index: 7692602 (Open Web Text)

Max Activating Token Index: 999