Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1122 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.6715. Min Act: -0.1700

Data Index: 3123245 (Open Web Text)

Max Activating Token Index: 709

Text #1

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.6667. Min Act: -0.1700

Data Index: 8033480 (Open Web Text)

Max Activating Token Index: 815

Text #2

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.6712. Min Act: -0.1700

Data Index: 786285 (Open Web Text)

Max Activating Token Index: 330

Text #3

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.6254. Min Act: -0.1700

Data Index: 2063619 (Open Web Text)

Max Activating Token Index: 330

Text #4

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.6340. Min Act: -0.1700

Data Index: 7045301 (Open Web Text)

Max Activating Token Index: 329

Text #5

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5816. Min Act: -0.1700

Data Index: 2114810 (Open Web Text)

Max Activating Token Index: 1022

Text #6

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5802. Min Act: -0.1700

Data Index: 1449006 (Open Web Text)

Max Activating Token Index: 772

Text #7

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5412. Min Act: -0.1700

Data Index: 7809020 (Open Web Text)

Max Activating Token Index: 346

Text #8

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5812. Min Act: -0.1700

Data Index: 7994787 (Open Web Text)

Max Activating Token Index: 452

Text #9

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5371. Min Act: -0.1700

Data Index: 3256150 (Open Web Text)

Max Activating Token Index: 283

Text #10

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5779. Min Act: -0.1700

Data Index: 3906130 (Open Web Text)

Max Activating Token Index: 376

Text #11

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5504. Min Act: -0.1700

Data Index: 1364414 (Open Web Text)

Max Activating Token Index: 314

Text #12

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5196. Min Act: -0.1700

Data Index: 3678649 (Open Web Text)

Max Activating Token Index: 725

Text #13

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5320. Min Act: -0.1700

Data Index: 7330997 (Open Web Text)

Max Activating Token Index: 892

Text #14

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5252. Min Act: -0.1700

Data Index: 659761 (Open Web Text)

Max Activating Token Index: 224

Text #15

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5257. Min Act: -0.1700

Data Index: 834053 (Open Web Text)

Max Activating Token Index: 948

Text #16

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5109. Min Act: -0.1700

Data Index: 7665956 (Open Web Text)

Max Activating Token Index: 227

Text #17

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5274. Min Act: -0.1700

Data Index: 4946290 (Open Web Text)

Max Activating Token Index: 758

Text #18

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5328. Min Act: -0.1700

Data Index: 5337784 (Open Web Text)

Max Activating Token Index: 886

Text #19

Max Range: 4.6715. Min Range: -4.6715

Max Act: 4.5029. Min Act: -0.1700

Data Index: 1488007 (Open Web Text)

Max Activating Token Index: 88