Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1138 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.5829. Min Act: -0.1700

Data Index: 6651648 (Open Web Text)

Max Activating Token Index: 36

Text #1

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.6038. Min Act: -0.1700

Data Index: 5912879 (Open Web Text)

Max Activating Token Index: 459

Text #2

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.5782. Min Act: -0.1700

Data Index: 1637170 (Open Web Text)

Max Activating Token Index: 390

Text #3

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.5457. Min Act: -0.1700

Data Index: 140807 (Open Web Text)

Max Activating Token Index: 759

Text #4

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.5755. Min Act: -0.1700

Data Index: 6261092 (Open Web Text)

Max Activating Token Index: 634

Text #5

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3998. Min Act: -0.1700

Data Index: 4548798 (Open Web Text)

Max Activating Token Index: 1020

Text #6

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.4051. Min Act: -0.1700

Data Index: 1832277 (Open Web Text)

Max Activating Token Index: 318

Text #7

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3813. Min Act: -0.1700

Data Index: 1989273 (Open Web Text)

Max Activating Token Index: 96

Text #8

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3680. Min Act: -0.1700

Data Index: 7463184 (Open Web Text)

Max Activating Token Index: 265

Text #9

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3883. Min Act: -0.1700

Data Index: 6530438 (Open Web Text)

Max Activating Token Index: 224

Text #10

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3768. Min Act: -0.1700

Data Index: 7669985 (Open Web Text)

Max Activating Token Index: 48

Text #11

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3561. Min Act: -0.1700

Data Index: 5548779 (Open Web Text)

Max Activating Token Index: 870

Text #12

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3495. Min Act: -0.1700

Data Index: 442131 (Open Web Text)

Max Activating Token Index: 228

Text #13

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3380. Min Act: -0.1700

Data Index: 6318556 (Open Web Text)

Max Activating Token Index: 549

Text #14

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3087. Min Act: -0.1700

Data Index: 1589841 (Open Web Text)

Max Activating Token Index: 717

Text #15

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.3330. Min Act: -0.1700

Data Index: 3716 (Open Web Text)

Max Activating Token Index: 456

Text #16

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.2835. Min Act: -0.1700

Data Index: 7076464 (Open Web Text)

Max Activating Token Index: 370

Text #17

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.2816. Min Act: -0.1700

Data Index: 1175398 (Open Web Text)

Max Activating Token Index: 459

Text #18

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.2470. Min Act: -0.1700

Data Index: 1316926 (Open Web Text)

Max Activating Token Index: 629

Text #19

Max Range: 4.6038. Min Range: -4.6038

Max Act: 4.2700. Min Act: -0.1700

Data Index: 442600 (Open Web Text)

Max Activating Token Index: 489