Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2465 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1721. Min Range: -3.1721

Max Act: 3.1721. Min Act: -0.1700

Data Index: 3429207 (Open Web Text)

Max Activating Token Index: 992

Text #1

Max Range: 3.1721. Min Range: -3.1721

Max Act: 3.1353. Min Act: -0.1700

Data Index: 2163691 (Open Web Text)

Max Activating Token Index: 113

Text #2

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.9741. Min Act: -0.1700

Data Index: 3646008 (Open Web Text)

Max Activating Token Index: 270

Text #3

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.9605. Min Act: -0.1700

Data Index: 8646075 (Open Web Text)

Max Activating Token Index: 646

Text #4

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.9068. Min Act: -0.1700

Data Index: 661729 (Open Web Text)

Max Activating Token Index: 540

Text #5

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.9136. Min Act: -0.1700

Data Index: 7206557 (Open Web Text)

Max Activating Token Index: 387

Text #6

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8984. Min Act: -0.1700

Data Index: 1317843 (Open Web Text)

Max Activating Token Index: 714

Text #7

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8888. Min Act: -0.1700

Data Index: 2549092 (Open Web Text)

Max Activating Token Index: 868

Text #8

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8852. Min Act: -0.1700

Data Index: 5340978 (Open Web Text)

Max Activating Token Index: 696

Text #9

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8669. Min Act: -0.1700

Data Index: 4686467 (Open Web Text)

Max Activating Token Index: 922

Text #10

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8766. Min Act: -0.1700

Data Index: 3932790 (Open Web Text)

Max Activating Token Index: 399

Text #11

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8588. Min Act: -0.1700

Data Index: 6355002 (Open Web Text)

Max Activating Token Index: 885

Text #12

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8721. Min Act: -0.1700

Data Index: 4331124 (Open Web Text)

Max Activating Token Index: 855

Text #13

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8792. Min Act: -0.1700

Data Index: 2486530 (Open Web Text)

Max Activating Token Index: 468

Text #14

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8421. Min Act: -0.1700

Data Index: 4034118 (Open Web Text)

Max Activating Token Index: 749

Text #15

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8490. Min Act: -0.1700

Data Index: 3683800 (Open Web Text)

Max Activating Token Index: 587

Text #16

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8168. Min Act: -0.1700

Data Index: 2904749 (Open Web Text)

Max Activating Token Index: 23

Text #17

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8205. Min Act: -0.1700

Data Index: 2963173 (Open Web Text)

Max Activating Token Index: 933

Text #18

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8433. Min Act: -0.1700

Data Index: 7630142 (Open Web Text)

Max Activating Token Index: 350

Text #19

Max Range: 3.1721. Min Range: -3.1721

Max Act: 2.8553. Min Act: -0.1700

Data Index: 6457189 (Open Web Text)

Max Activating Token Index: 452