Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2081 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7848. Min Act: -0.1700

Data Index: 7930504 (Open Web Text)

Max Activating Token Index: 878

Text #1

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7452. Min Act: -0.1700

Data Index: 3591678 (Open Web Text)

Max Activating Token Index: 497

Text #2

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7390. Min Act: -0.1700

Data Index: 6051289 (Open Web Text)

Max Activating Token Index: 352

Text #3

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7384. Min Act: -0.1700

Data Index: 5729900 (Open Web Text)

Max Activating Token Index: 685

Text #4

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7316. Min Act: -0.1700

Data Index: 1623092 (Open Web Text)

Max Activating Token Index: 844

Text #5

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7356. Min Act: -0.1700

Data Index: 4976070 (Open Web Text)

Max Activating Token Index: 193

Text #6

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7077. Min Act: -0.1700

Data Index: 4980876 (Open Web Text)

Max Activating Token Index: 993

Text #7

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7056. Min Act: -0.1700

Data Index: 3510195 (Open Web Text)

Max Activating Token Index: 524

Text #8

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7097. Min Act: -0.1700

Data Index: 5639757 (Open Web Text)

Max Activating Token Index: 700

Text #9

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7115. Min Act: -0.1700

Data Index: 200135 (Open Web Text)

Max Activating Token Index: 117

Text #10

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6996. Min Act: -0.1700

Data Index: 8185350 (Open Web Text)

Max Activating Token Index: 885

Text #11

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6842. Min Act: -0.1700

Data Index: 7151209 (Open Web Text)

Max Activating Token Index: 192

Text #12

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6995. Min Act: -0.1700

Data Index: 8636584 (Open Web Text)

Max Activating Token Index: 537

Text #13

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6984. Min Act: -0.1700

Data Index: 7184111 (Open Web Text)

Max Activating Token Index: 622

Text #14

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7031. Min Act: -0.1700

Data Index: 1270400 (Open Web Text)

Max Activating Token Index: 501

Text #15

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.7020. Min Act: -0.1700

Data Index: 986069 (Open Web Text)

Max Activating Token Index: 745

Text #16

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6815. Min Act: -0.1700

Data Index: 4166237 (Open Web Text)

Max Activating Token Index: 606

Text #17

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6964. Min Act: -0.1700

Data Index: 5339742 (Open Web Text)

Max Activating Token Index: 194

Text #18

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6909. Min Act: -0.1700

Data Index: 3577134 (Open Web Text)

Max Activating Token Index: 373

Text #19

Max Range: 1.7848. Min Range: -1.7848

Max Act: 1.6746. Min Act: -0.1700

Data Index: 2243456 (Open Web Text)

Max Activating Token Index: 846