Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 72 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.6310. Min Act: -0.1700

Data Index: 7169167 (Open Web Text)

Max Activating Token Index: 763

Text #1

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.5701. Min Act: -0.1700

Data Index: 7921908 (Open Web Text)

Max Activating Token Index: 825

Text #2

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.5067. Min Act: -0.1700

Data Index: 2729926 (Open Web Text)

Max Activating Token Index: 463

Text #3

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.4978. Min Act: -0.1700

Data Index: 1248568 (Open Web Text)

Max Activating Token Index: 981

Text #4

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3394. Min Act: -0.1700

Data Index: 3394717 (Open Web Text)

Max Activating Token Index: 991

Text #5

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3698. Min Act: -0.1700

Data Index: 2109480 (Open Web Text)

Max Activating Token Index: 976

Text #6

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3559. Min Act: -0.1700

Data Index: 5287143 (Open Web Text)

Max Activating Token Index: 99

Text #7

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3272. Min Act: -0.1700

Data Index: 3305745 (Open Web Text)

Max Activating Token Index: 894

Text #8

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3095. Min Act: -0.1700

Data Index: 1394162 (Open Web Text)

Max Activating Token Index: 530

Text #9

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3303. Min Act: -0.1700

Data Index: 728881 (Open Web Text)

Max Activating Token Index: 697

Text #10

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3015. Min Act: -0.1700

Data Index: 7412884 (Open Web Text)

Max Activating Token Index: 950

Text #11

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2934. Min Act: -0.1700

Data Index: 3224919 (Open Web Text)

Max Activating Token Index: 340

Text #12

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2939. Min Act: -0.1700

Data Index: 6935530 (Open Web Text)

Max Activating Token Index: 421

Text #13

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.3122. Min Act: -0.1700

Data Index: 1579100 (Open Web Text)

Max Activating Token Index: 500

Text #14

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2801. Min Act: -0.1700

Data Index: 7360307 (Open Web Text)

Max Activating Token Index: 586

Text #15

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2976. Min Act: -0.1700

Data Index: 5922035 (Open Web Text)

Max Activating Token Index: 680

Text #16

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2880. Min Act: -0.1700

Data Index: 8642025 (Open Web Text)

Max Activating Token Index: 525

Text #17

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2381. Min Act: -0.1700

Data Index: 54214 (Open Web Text)

Max Activating Token Index: 269

Text #18

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2237. Min Act: -0.1700

Data Index: 101033 (Open Web Text)

Max Activating Token Index: 532

Text #19

Max Range: 4.6310. Min Range: -4.6310

Max Act: 4.2421. Min Act: -0.1700

Data Index: 124769 (Open Web Text)

Max Activating Token Index: 567