Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2793 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.9412. Min Act: -0.1700

Data Index: 8483416 (Open Web Text)

Max Activating Token Index: 260

Text #1

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.8044. Min Act: -0.1700

Data Index: 1049456 (Open Web Text)

Max Activating Token Index: 396

Text #2

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6776. Min Act: -0.1700

Data Index: 3683193 (Open Web Text)

Max Activating Token Index: 162

Text #3

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6658. Min Act: -0.1700

Data Index: 5947774 (Open Web Text)

Max Activating Token Index: 785

Text #4

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.7188. Min Act: -0.1700

Data Index: 4342781 (Open Web Text)

Max Activating Token Index: 139

Text #5

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.7089. Min Act: -0.1700

Data Index: 6461406 (Open Web Text)

Max Activating Token Index: 16

Text #6

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6532. Min Act: -0.1700

Data Index: 4864392 (Open Web Text)

Max Activating Token Index: 132

Text #7

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6543. Min Act: -0.1700

Data Index: 6866118 (Open Web Text)

Max Activating Token Index: 46

Text #8

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6114. Min Act: -0.1700

Data Index: 6923968 (Open Web Text)

Max Activating Token Index: 32

Text #9

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6313. Min Act: -0.1700

Data Index: 5253939 (Open Web Text)

Max Activating Token Index: 747

Text #10

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6149. Min Act: -0.1700

Data Index: 5548761 (Open Web Text)

Max Activating Token Index: 938

Text #11

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6058. Min Act: -0.1700

Data Index: 1417206 (Open Web Text)

Max Activating Token Index: 167

Text #12

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.5982. Min Act: -0.1700

Data Index: 4030192 (Open Web Text)

Max Activating Token Index: 168

Text #13

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6016. Min Act: -0.1700

Data Index: 6247659 (Open Web Text)

Max Activating Token Index: 898

Text #14

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.5944. Min Act: -0.1700

Data Index: 7111764 (Open Web Text)

Max Activating Token Index: 116

Text #15

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.6067. Min Act: -0.1700

Data Index: 6788417 (Open Web Text)

Max Activating Token Index: 291

Text #16

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.5491. Min Act: -0.1700

Data Index: 3019600 (Open Web Text)

Max Activating Token Index: 613

Text #17

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.5678. Min Act: -0.1700

Data Index: 3014899 (Open Web Text)

Max Activating Token Index: 410

Text #18

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.5776. Min Act: -0.1700

Data Index: 836411 (Open Web Text)

Max Activating Token Index: 670

Text #19

Max Range: 3.9412. Min Range: -3.9412

Max Act: 3.5560. Min Act: -0.1700

Data Index: 1989969 (Open Web Text)

Max Activating Token Index: 393