Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2033 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.1965. Min Act: -0.1700

Data Index: 2809013 (Open Web Text)

Max Activating Token Index: 504

Text #1

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.1547. Min Act: -0.1700

Data Index: 8413538 (Open Web Text)

Max Activating Token Index: 446

Text #2

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.1489. Min Act: -0.1700

Data Index: 994735 (Open Web Text)

Max Activating Token Index: 245

Text #3

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.1361. Min Act: -0.1700

Data Index: 4662492 (Open Web Text)

Max Activating Token Index: 228

Text #4

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.1083. Min Act: -0.1700

Data Index: 2732900 (Open Web Text)

Max Activating Token Index: 476

Text #5

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.1036. Min Act: -0.1700

Data Index: 6965667 (Open Web Text)

Max Activating Token Index: 364

Text #6

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0968. Min Act: -0.1700

Data Index: 4178571 (Open Web Text)

Max Activating Token Index: 942

Text #7

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0904. Min Act: -0.1700

Data Index: 1466501 (Open Web Text)

Max Activating Token Index: 735

Text #8

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0923. Min Act: -0.1700

Data Index: 1533294 (Open Web Text)

Max Activating Token Index: 1000

Text #9

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0849. Min Act: -0.1700

Data Index: 2783123 (Open Web Text)

Max Activating Token Index: 186

Text #10

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0719. Min Act: -0.1700

Data Index: 4386524 (Open Web Text)

Max Activating Token Index: 358

Text #11

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0868. Min Act: -0.1700

Data Index: 517855 (Open Web Text)

Max Activating Token Index: 533

Text #12

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0676. Min Act: -0.1700

Data Index: 8456226 (Open Web Text)

Max Activating Token Index: 530

Text #13

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0712. Min Act: -0.1700

Data Index: 146625 (Open Web Text)

Max Activating Token Index: 543

Text #14

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0555. Min Act: -0.1700

Data Index: 5778877 (Open Web Text)

Max Activating Token Index: 469

Text #15

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0434. Min Act: -0.1700

Data Index: 551791 (Open Web Text)

Max Activating Token Index: 180

Text #16

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0494. Min Act: -0.1700

Data Index: 5546744 (Open Web Text)

Max Activating Token Index: 118

Text #17

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0606. Min Act: -0.1700

Data Index: 3315044 (Open Web Text)

Max Activating Token Index: 124

Text #18

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0561. Min Act: -0.1700

Data Index: 1566527 (Open Web Text)

Max Activating Token Index: 242

Text #19

Max Range: 2.1965. Min Range: -2.1965

Max Act: 2.0646. Min Act: -0.1700

Data Index: 3740002 (Open Web Text)

Max Activating Token Index: 141