Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2630 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9563. Min Act: -0.1700

Data Index: 4376434 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9696. Min Act: -0.1700

Data Index: 1944079 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9665. Min Act: -0.1700

Data Index: 6800402 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9366. Min Act: -0.1700

Data Index: 6359947 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9195. Min Act: -0.1700

Data Index: 6642229 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9068. Min Act: -0.1700

Data Index: 4569190 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9366. Min Act: -0.1700

Data Index: 4060833 (Open Web Text)

Max Activating Token Index: 12

Text #7

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.9205. Min Act: -0.1700

Data Index: 8738470 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8978. Min Act: -0.1700

Data Index: 518303 (Open Web Text)

Max Activating Token Index: 14

Text #9

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8874. Min Act: -0.1700

Data Index: 7074736 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8794. Min Act: -0.1700

Data Index: 2681573 (Open Web Text)

Max Activating Token Index: 77

Text #11

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8861. Min Act: -0.1700

Data Index: 2607666 (Open Web Text)

Max Activating Token Index: 947

Text #12

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8684. Min Act: -0.1700

Data Index: 1776859 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8514. Min Act: -0.1700

Data Index: 5631224 (Open Web Text)

Max Activating Token Index: 12

Text #14

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8625. Min Act: -0.1700

Data Index: 2706435 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8739. Min Act: -0.1700

Data Index: 5663455 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8605. Min Act: -0.1700

Data Index: 2939231 (Open Web Text)

Max Activating Token Index: 103

Text #17

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8572. Min Act: -0.1700

Data Index: 827044 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8659. Min Act: -0.1700

Data Index: 6242475 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 3.9696. Min Range: -3.9696

Max Act: 3.8559. Min Act: -0.1700

Data Index: 2155080 (Open Web Text)

Max Activating Token Index: 6