Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1981 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.5742. Min Act: -0.1700

Data Index: 1561705 (Open Web Text)

Max Activating Token Index: 919

Text #1

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.5248. Min Act: -0.1700

Data Index: 6387449 (Open Web Text)

Max Activating Token Index: 382

Text #2

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.5031. Min Act: -0.1700

Data Index: 6328530 (Open Web Text)

Max Activating Token Index: 635

Text #3

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4709. Min Act: -0.1700

Data Index: 8629795 (Open Web Text)

Max Activating Token Index: 34

Text #4

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4866. Min Act: -0.1700

Data Index: 5479551 (Open Web Text)

Max Activating Token Index: 712

Text #5

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4771. Min Act: -0.1700

Data Index: 1215567 (Open Web Text)

Max Activating Token Index: 545

Text #6

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4560. Min Act: -0.1700

Data Index: 7421891 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4589. Min Act: -0.1700

Data Index: 3985343 (Open Web Text)

Max Activating Token Index: 489

Text #8

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4492. Min Act: -0.1700

Data Index: 4994781 (Open Web Text)

Max Activating Token Index: 347

Text #9

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4518. Min Act: -0.1700

Data Index: 4751724 (Open Web Text)

Max Activating Token Index: 432

Text #10

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4423. Min Act: -0.1700

Data Index: 3819540 (Open Web Text)

Max Activating Token Index: 550

Text #11

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4294. Min Act: -0.1700

Data Index: 2834861 (Open Web Text)

Max Activating Token Index: 346

Text #12

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4308. Min Act: -0.1700

Data Index: 5616192 (Open Web Text)

Max Activating Token Index: 284

Text #13

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4357. Min Act: -0.1700

Data Index: 6703187 (Open Web Text)

Max Activating Token Index: 514

Text #14

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4407. Min Act: -0.1700

Data Index: 6294166 (Open Web Text)

Max Activating Token Index: 510

Text #15

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4256. Min Act: -0.1700

Data Index: 4388666 (Open Web Text)

Max Activating Token Index: 257

Text #16

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4251. Min Act: -0.1700

Data Index: 1497846 (Open Web Text)

Max Activating Token Index: 873

Text #17

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4074. Min Act: -0.1700

Data Index: 4627882 (Open Web Text)

Max Activating Token Index: 464

Text #18

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.4073. Min Act: -0.1700

Data Index: 5118652 (Open Web Text)

Max Activating Token Index: 517

Text #19

Max Range: 1.5742. Min Range: -1.5742

Max Act: 1.3974. Min Act: -0.1700

Data Index: 4941566 (Open Web Text)

Max Activating Token Index: 5