Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2396 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.6970. Min Act: -0.1700

Data Index: 2953896 (Open Web Text)

Max Activating Token Index: 593

Text #1

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.7020. Min Act: -0.1700

Data Index: 3046630 (Open Web Text)

Max Activating Token Index: 615

Text #2

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.6214. Min Act: -0.1700

Data Index: 4610130 (Open Web Text)

Max Activating Token Index: 510

Text #3

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.6382. Min Act: -0.1700

Data Index: 7009719 (Open Web Text)

Max Activating Token Index: 903

Text #4

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.6027. Min Act: -0.1700

Data Index: 4341334 (Open Web Text)

Max Activating Token Index: 32

Text #5

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5820. Min Act: -0.1700

Data Index: 1203254 (Open Web Text)

Max Activating Token Index: 773

Text #6

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5573. Min Act: -0.1700

Data Index: 1401863 (Open Web Text)

Max Activating Token Index: 542

Text #7

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5779. Min Act: -0.1700

Data Index: 6486603 (Open Web Text)

Max Activating Token Index: 246

Text #8

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5329. Min Act: -0.1700

Data Index: 6217349 (Open Web Text)

Max Activating Token Index: 162

Text #9

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5468. Min Act: -0.1700

Data Index: 4895665 (Open Web Text)

Max Activating Token Index: 77

Text #10

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5265. Min Act: -0.1700

Data Index: 6903664 (Open Web Text)

Max Activating Token Index: 726

Text #11

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5614. Min Act: -0.1700

Data Index: 7392776 (Open Web Text)

Max Activating Token Index: 728

Text #12

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5324. Min Act: -0.1700

Data Index: 3141072 (Open Web Text)

Max Activating Token Index: 493

Text #13

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5316. Min Act: -0.1700

Data Index: 5056642 (Open Web Text)

Max Activating Token Index: 438

Text #14

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5006. Min Act: -0.1700

Data Index: 7689209 (Open Web Text)

Max Activating Token Index: 993

Text #15

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5120. Min Act: -0.1700

Data Index: 7501723 (Open Web Text)

Max Activating Token Index: 436

Text #16

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5068. Min Act: -0.1700

Data Index: 5805014 (Open Web Text)

Max Activating Token Index: 447

Text #17

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5069. Min Act: -0.1700

Data Index: 973657 (Open Web Text)

Max Activating Token Index: 180

Text #18

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5067. Min Act: -0.1700

Data Index: 3027998 (Open Web Text)

Max Activating Token Index: 50

Text #19

Max Range: 4.7020. Min Range: -4.7020

Max Act: 4.5147. Min Act: -0.1700

Data Index: 3925906 (Open Web Text)

Max Activating Token Index: 952