Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1685 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2593. Min Range: -5.2593

Max Act: 5.2593. Min Act: -0.1700

Data Index: 6224615 (Open Web Text)

Max Activating Token Index: 706

Text #1

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.9634. Min Act: -0.1700

Data Index: 7350945 (Open Web Text)

Max Activating Token Index: 1020

Text #2

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.8647. Min Act: -0.1700

Data Index: 2335579 (Open Web Text)

Max Activating Token Index: 477

Text #3

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.8396. Min Act: -0.1700

Data Index: 232555 (Open Web Text)

Max Activating Token Index: 811

Text #4

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.8011. Min Act: -0.1700

Data Index: 2706422 (Open Web Text)

Max Activating Token Index: 429

Text #5

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.7988. Min Act: -0.1700

Data Index: 4131120 (Open Web Text)

Max Activating Token Index: 804

Text #6

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.8107. Min Act: -0.1700

Data Index: 7018896 (Open Web Text)

Max Activating Token Index: 750

Text #7

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.8065. Min Act: -0.1700

Data Index: 2928282 (Open Web Text)

Max Activating Token Index: 931

Text #8

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.7850. Min Act: -0.1700

Data Index: 4947362 (Open Web Text)

Max Activating Token Index: 502

Text #9

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.7727. Min Act: -0.1700

Data Index: 7262491 (Open Web Text)

Max Activating Token Index: 95

Text #10

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6850. Min Act: -0.1700

Data Index: 6781326 (Open Web Text)

Max Activating Token Index: 490

Text #11

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6751. Min Act: -0.1700

Data Index: 1592213 (Open Web Text)

Max Activating Token Index: 735

Text #12

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6995. Min Act: -0.1700

Data Index: 2052855 (Open Web Text)

Max Activating Token Index: 310

Text #13

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6848. Min Act: -0.1700

Data Index: 4043864 (Open Web Text)

Max Activating Token Index: 407

Text #14

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6650. Min Act: -0.1700

Data Index: 3308268 (Open Web Text)

Max Activating Token Index: 967

Text #15

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6732. Min Act: -0.1700

Data Index: 4902438 (Open Web Text)

Max Activating Token Index: 318

Text #16

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6661. Min Act: -0.1700

Data Index: 5576236 (Open Web Text)

Max Activating Token Index: 776

Text #17

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6471. Min Act: -0.1700

Data Index: 2043418 (Open Web Text)

Max Activating Token Index: 794

Text #18

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6627. Min Act: -0.1700

Data Index: 1927851 (Open Web Text)

Max Activating Token Index: 439

Text #19

Max Range: 5.2593. Min Range: -5.2593

Max Act: 4.6855. Min Act: -0.1700

Data Index: 7983606 (Open Web Text)

Max Activating Token Index: 590