Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1920 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.4310. Min Act: -0.1700

Data Index: 928732 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.4341. Min Act: -0.1700

Data Index: 7717360 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.4055. Min Act: -0.1700

Data Index: 4709776 (Open Web Text)

Max Activating Token Index: 872

Text #3

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3972. Min Act: -0.1700

Data Index: 6417248 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3920. Min Act: -0.1700

Data Index: 1167945 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3903. Min Act: -0.1700

Data Index: 8699901 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3920. Min Act: -0.1700

Data Index: 915560 (Open Web Text)

Max Activating Token Index: 22

Text #7

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3805. Min Act: -0.1700

Data Index: 4238471 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3738. Min Act: -0.1700

Data Index: 2467473 (Open Web Text)

Max Activating Token Index: 1015

Text #9

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3607. Min Act: -0.1700

Data Index: 908117 (Open Web Text)

Max Activating Token Index: 33

Text #10

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3848. Min Act: -0.1700

Data Index: 3371995 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3708. Min Act: -0.1700

Data Index: 8361364 (Open Web Text)

Max Activating Token Index: 997

Text #12

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3493. Min Act: -0.1700

Data Index: 7805047 (Open Web Text)

Max Activating Token Index: 1007

Text #13

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3715. Min Act: -0.1700

Data Index: 6555909 (Open Web Text)

Max Activating Token Index: 2

Text #14

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3556. Min Act: -0.1700

Data Index: 2887375 (Open Web Text)

Max Activating Token Index: 1019

Text #15

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3704. Min Act: -0.1700

Data Index: 1874238 (Open Web Text)

Max Activating Token Index: 12

Text #16

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3347. Min Act: -0.1700

Data Index: 4376007 (Open Web Text)

Max Activating Token Index: 865

Text #17

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3381. Min Act: -0.1700

Data Index: 4740606 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3348. Min Act: -0.1700

Data Index: 5157936 (Open Web Text)

Max Activating Token Index: 16

Text #19

Max Range: 3.4341. Min Range: -3.4341

Max Act: 3.3330. Min Act: -0.1700

Data Index: 5933714 (Open Web Text)

Max Activating Token Index: 791