Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 212 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.4642. Min Act: -0.1700

Data Index: 2606282 (Open Web Text)

Max Activating Token Index: 631

Text #1

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.4170. Min Act: -0.1700

Data Index: 7518739 (Open Web Text)

Max Activating Token Index: 839

Text #2

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3976. Min Act: -0.1700

Data Index: 7632246 (Open Web Text)

Max Activating Token Index: 943

Text #3

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3959. Min Act: -0.1700

Data Index: 7640581 (Open Web Text)

Max Activating Token Index: 867

Text #4

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.4015. Min Act: -0.1700

Data Index: 416567 (Open Web Text)

Max Activating Token Index: 472

Text #5

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3759. Min Act: -0.1700

Data Index: 729336 (Open Web Text)

Max Activating Token Index: 279

Text #6

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3549. Min Act: -0.1700

Data Index: 359956 (Open Web Text)

Max Activating Token Index: 647

Text #7

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3313. Min Act: -0.1700

Data Index: 5430667 (Open Web Text)

Max Activating Token Index: 729

Text #8

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3486. Min Act: -0.1700

Data Index: 2480081 (Open Web Text)

Max Activating Token Index: 586

Text #9

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3335. Min Act: -0.1700

Data Index: 7356675 (Open Web Text)

Max Activating Token Index: 970

Text #10

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3357. Min Act: -0.1700

Data Index: 7825135 (Open Web Text)

Max Activating Token Index: 593

Text #11

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3231. Min Act: -0.1700

Data Index: 3430673 (Open Web Text)

Max Activating Token Index: 843

Text #12

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3139. Min Act: -0.1700

Data Index: 3165107 (Open Web Text)

Max Activating Token Index: 599

Text #13

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3059. Min Act: -0.1700

Data Index: 3390960 (Open Web Text)

Max Activating Token Index: 632

Text #14

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.2992. Min Act: -0.1700

Data Index: 611223 (Open Web Text)

Max Activating Token Index: 196

Text #15

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.3095. Min Act: -0.1700

Data Index: 7610810 (Open Web Text)

Max Activating Token Index: 631

Text #16

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.2997. Min Act: -0.1700

Data Index: 155231 (Open Web Text)

Max Activating Token Index: 102

Text #17

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.2856. Min Act: -0.1700

Data Index: 7516359 (Open Web Text)

Max Activating Token Index: 349

Text #18

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.2727. Min Act: -0.1700

Data Index: 4181728 (Open Web Text)

Max Activating Token Index: 910

Text #19

Max Range: 3.4642. Min Range: -3.4642

Max Act: 3.2789. Min Act: -0.1700

Data Index: 2467973 (Open Web Text)

Max Activating Token Index: 772