Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1123 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.7423. Min Act: -0.1700

Data Index: 6942170 (Open Web Text)

Max Activating Token Index: 583

Text #1

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.6708. Min Act: -0.1700

Data Index: 5156984 (Open Web Text)

Max Activating Token Index: 225

Text #2

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.5564. Min Act: -0.1700

Data Index: 790977 (Open Web Text)

Max Activating Token Index: 112

Text #3

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.5281. Min Act: -0.1700

Data Index: 1572126 (Open Web Text)

Max Activating Token Index: 731

Text #4

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.5348. Min Act: -0.1700

Data Index: 3080550 (Open Web Text)

Max Activating Token Index: 822

Text #5

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.5166. Min Act: -0.1700

Data Index: 6683967 (Open Web Text)

Max Activating Token Index: 559

Text #6

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4975. Min Act: -0.1700

Data Index: 8757242 (Open Web Text)

Max Activating Token Index: 399

Text #7

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.5082. Min Act: -0.1700

Data Index: 8311474 (Open Web Text)

Max Activating Token Index: 319

Text #8

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4901. Min Act: -0.1700

Data Index: 3664475 (Open Web Text)

Max Activating Token Index: 493

Text #9

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4890. Min Act: -0.1700

Data Index: 4962809 (Open Web Text)

Max Activating Token Index: 791

Text #10

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4754. Min Act: -0.1700

Data Index: 2442908 (Open Web Text)

Max Activating Token Index: 256

Text #11

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.5023. Min Act: -0.1700

Data Index: 4859274 (Open Web Text)

Max Activating Token Index: 374

Text #12

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4814. Min Act: -0.1700

Data Index: 1658095 (Open Web Text)

Max Activating Token Index: 773

Text #13

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4904. Min Act: -0.1700

Data Index: 8796722 (Open Web Text)

Max Activating Token Index: 454

Text #14

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4700. Min Act: -0.1700

Data Index: 7954308 (Open Web Text)

Max Activating Token Index: 434

Text #15

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4649. Min Act: -0.1700

Data Index: 5602369 (Open Web Text)

Max Activating Token Index: 89

Text #16

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4371. Min Act: -0.1700

Data Index: 5081602 (Open Web Text)

Max Activating Token Index: 392

Text #17

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4589. Min Act: -0.1700

Data Index: 3029606 (Open Web Text)

Max Activating Token Index: 957

Text #18

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4631. Min Act: -0.1700

Data Index: 1658096 (Open Web Text)

Max Activating Token Index: 578

Text #19

Max Range: 3.7423. Min Range: -3.7423

Max Act: 3.4427. Min Act: -0.1700

Data Index: 6417869 (Open Web Text)

Max Activating Token Index: 973