Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1755 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1167. Min Range: -3.1167

Max Act: 3.1167. Min Act: -0.1700

Data Index: 1360172 (Open Web Text)

Max Activating Token Index: 302

Text #1

Max Range: 3.1167. Min Range: -3.1167

Max Act: 3.0058. Min Act: -0.1700

Data Index: 271860 (Open Web Text)

Max Activating Token Index: 432

Text #2

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.9471. Min Act: -0.1700

Data Index: 956196 (Open Web Text)

Max Activating Token Index: 105

Text #3

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.9504. Min Act: -0.1700

Data Index: 4934576 (Open Web Text)

Max Activating Token Index: 449

Text #4

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.9210. Min Act: -0.1700

Data Index: 4104391 (Open Web Text)

Max Activating Token Index: 201

Text #5

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.9061. Min Act: -0.1700

Data Index: 4280803 (Open Web Text)

Max Activating Token Index: 747

Text #6

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.8648. Min Act: -0.1700

Data Index: 4529731 (Open Web Text)

Max Activating Token Index: 107

Text #7

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.8526. Min Act: -0.1700

Data Index: 5286535 (Open Web Text)

Max Activating Token Index: 368

Text #8

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7995. Min Act: -0.1700

Data Index: 6202294 (Open Web Text)

Max Activating Token Index: 864

Text #9

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.8205. Min Act: -0.1700

Data Index: 1291364 (Open Web Text)

Max Activating Token Index: 511

Text #10

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.8010. Min Act: -0.1700

Data Index: 5731698 (Open Web Text)

Max Activating Token Index: 23

Text #11

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.8100. Min Act: -0.1700

Data Index: 899501 (Open Web Text)

Max Activating Token Index: 631

Text #12

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7682. Min Act: -0.1700

Data Index: 4856146 (Open Web Text)

Max Activating Token Index: 770

Text #13

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7924. Min Act: -0.1700

Data Index: 5636703 (Open Web Text)

Max Activating Token Index: 587

Text #14

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7693. Min Act: -0.1700

Data Index: 6048191 (Open Web Text)

Max Activating Token Index: 229

Text #15

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7767. Min Act: -0.1700

Data Index: 4112275 (Open Web Text)

Max Activating Token Index: 120

Text #16

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7692. Min Act: -0.1700

Data Index: 3165706 (Open Web Text)

Max Activating Token Index: 609

Text #17

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7452. Min Act: -0.1700

Data Index: 4885152 (Open Web Text)

Max Activating Token Index: 483

Text #18

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7607. Min Act: -0.1700

Data Index: 247782 (Open Web Text)

Max Activating Token Index: 295

Text #19

Max Range: 3.1167. Min Range: -3.1167

Max Act: 2.7659. Min Act: -0.1700

Data Index: 2803207 (Open Web Text)

Max Activating Token Index: 420