Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1072 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.8089. Min Act: -0.1700

Data Index: 7713914 (Open Web Text)

Max Activating Token Index: 718

Text #1

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.8070. Min Act: -0.1683

Data Index: 2776478 (Open Web Text)

Max Activating Token Index: 48

Text #2

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7696. Min Act: -0.1698

Data Index: 3951022 (Open Web Text)

Max Activating Token Index: 169

Text #3

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7586. Min Act: -0.1700

Data Index: 8429791 (Open Web Text)

Max Activating Token Index: 337

Text #4

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7491. Min Act: -0.1700

Data Index: 3449211 (Open Web Text)

Max Activating Token Index: 387

Text #5

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7607. Min Act: -0.1700

Data Index: 4582704 (Open Web Text)

Max Activating Token Index: 967

Text #6

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7440. Min Act: -0.1700

Data Index: 2340729 (Open Web Text)

Max Activating Token Index: 449

Text #7

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7324. Min Act: -0.1700

Data Index: 685831 (Open Web Text)

Max Activating Token Index: 524

Text #8

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7355. Min Act: -0.1700

Data Index: 7087061 (Open Web Text)

Max Activating Token Index: 384

Text #9

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7414. Min Act: -0.1700

Data Index: 648353 (Open Web Text)

Max Activating Token Index: 401

Text #10

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7241. Min Act: -0.1696

Data Index: 2536805 (Open Web Text)

Max Activating Token Index: 624

Text #11

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7282. Min Act: -0.1700

Data Index: 8725759 (Open Web Text)

Max Activating Token Index: 297

Text #12

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7255. Min Act: -0.1700

Data Index: 2793514 (Open Web Text)

Max Activating Token Index: 992

Text #13

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7092. Min Act: -0.1700

Data Index: 5137285 (Open Web Text)

Max Activating Token Index: 144

Text #14

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7136. Min Act: -0.1700

Data Index: 2821613 (Open Web Text)

Max Activating Token Index: 85

Text #15

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7091. Min Act: -0.1700

Data Index: 4582705 (Open Web Text)

Max Activating Token Index: 44

Text #16

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7101. Min Act: -0.1700

Data Index: 7382843 (Open Web Text)

Max Activating Token Index: 120

Text #17

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.7102. Min Act: -0.1697

Data Index: 3828234 (Open Web Text)

Max Activating Token Index: 200

Text #18

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.6996. Min Act: -0.1700

Data Index: 4843009 (Open Web Text)

Max Activating Token Index: 484

Text #19

Max Range: 1.8089. Min Range: -1.8089

Max Act: 1.6859. Min Act: -0.1700

Data Index: 1988319 (Open Web Text)

Max Activating Token Index: 135