Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 692 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.2727. Min Act: -0.1700

Data Index: 5012894 (Open Web Text)

Max Activating Token Index: 590

Text #1

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.1468. Min Act: -0.1700

Data Index: 8210622 (Open Web Text)

Max Activating Token Index: 414

Text #2

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0870. Min Act: -0.1700

Data Index: 4848510 (Open Web Text)

Max Activating Token Index: 994

Text #3

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0469. Min Act: -0.1700

Data Index: 5207291 (Open Web Text)

Max Activating Token Index: 977

Text #4

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0669. Min Act: -0.1700

Data Index: 3897721 (Open Web Text)

Max Activating Token Index: 627

Text #5

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0238. Min Act: -0.1700

Data Index: 819767 (Open Web Text)

Max Activating Token Index: 329

Text #6

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0069. Min Act: -0.1700

Data Index: 6605379 (Open Web Text)

Max Activating Token Index: 966

Text #7

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0333. Min Act: -0.1700

Data Index: 7657238 (Open Web Text)

Max Activating Token Index: 381

Text #8

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0373. Min Act: -0.1700

Data Index: 2398166 (Open Web Text)

Max Activating Token Index: 791

Text #9

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0247. Min Act: -0.1700

Data Index: 2431492 (Open Web Text)

Max Activating Token Index: 493

Text #10

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0140. Min Act: -0.1700

Data Index: 5623534 (Open Web Text)

Max Activating Token Index: 962

Text #11

Max Range: 5.2727. Min Range: -5.2727

Max Act: 4.9743. Min Act: -0.1700

Data Index: 1728209 (Open Web Text)

Max Activating Token Index: 213

Text #12

Max Range: 5.2727. Min Range: -5.2727

Max Act: 4.9949. Min Act: -0.1700

Data Index: 2124063 (Open Web Text)

Max Activating Token Index: 883

Text #13

Max Range: 5.2727. Min Range: -5.2727

Max Act: 4.9936. Min Act: -0.1700

Data Index: 5776645 (Open Web Text)

Max Activating Token Index: 527

Text #14

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0268. Min Act: -0.1700

Data Index: 1998879 (Open Web Text)

Max Activating Token Index: 836

Text #15

Max Range: 5.2727. Min Range: -5.2727

Max Act: 4.9946. Min Act: -0.1700

Data Index: 1059453 (Open Web Text)

Max Activating Token Index: 448

Text #16

Max Range: 5.2727. Min Range: -5.2727

Max Act: 4.9837. Min Act: -0.1700

Data Index: 6230217 (Open Web Text)

Max Activating Token Index: 949

Text #17

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0037. Min Act: -0.1700

Data Index: 8625381 (Open Web Text)

Max Activating Token Index: 1006

Text #18

Max Range: 5.2727. Min Range: -5.2727

Max Act: 5.0060. Min Act: -0.1700

Data Index: 5825735 (Open Web Text)

Max Activating Token Index: 892

Text #19

Max Range: 5.2727. Min Range: -5.2727

Max Act: 4.9624. Min Act: -0.1700

Data Index: 6592581 (Open Web Text)

Max Activating Token Index: 615