Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 928 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.5864. Min Act: -0.1700

Data Index: 6388676 (Open Web Text)

Max Activating Token Index: 627

Text #1

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.5851. Min Act: -0.1700

Data Index: 6070822 (Open Web Text)

Max Activating Token Index: 753

Text #2

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.5363. Min Act: -0.1700

Data Index: 875778 (Open Web Text)

Max Activating Token Index: 616

Text #3

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.5389. Min Act: -0.1700

Data Index: 188754 (Open Web Text)

Max Activating Token Index: 849

Text #4

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.5362. Min Act: -0.1700

Data Index: 2955411 (Open Web Text)

Max Activating Token Index: 925

Text #5

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.4064. Min Act: -0.1700

Data Index: 6880753 (Open Web Text)

Max Activating Token Index: 761

Text #6

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.3909. Min Act: -0.1700

Data Index: 6223412 (Open Web Text)

Max Activating Token Index: 922

Text #7

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.3573. Min Act: -0.1700

Data Index: 4565652 (Open Web Text)

Max Activating Token Index: 980

Text #8

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.3993. Min Act: -0.1700

Data Index: 6173188 (Open Web Text)

Max Activating Token Index: 35

Text #9

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.3770. Min Act: -0.1695

Data Index: 4164442 (Open Web Text)

Max Activating Token Index: 844

Text #10

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.3614. Min Act: -0.1700

Data Index: 1237277 (Open Web Text)

Max Activating Token Index: 625

Text #11

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.3458. Min Act: -0.1700

Data Index: 6397242 (Open Web Text)

Max Activating Token Index: 757

Text #12

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.2646. Min Act: -0.1700

Data Index: 4068811 (Open Web Text)

Max Activating Token Index: 263

Text #13

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.2667. Min Act: -0.1700

Data Index: 5615628 (Open Web Text)

Max Activating Token Index: 965

Text #14

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.2359. Min Act: -0.1700

Data Index: 654849 (Open Web Text)

Max Activating Token Index: 444

Text #15

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.2419. Min Act: -0.1700

Data Index: 2885954 (Open Web Text)

Max Activating Token Index: 808

Text #16

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.2332. Min Act: -0.1700

Data Index: 4412736 (Open Web Text)

Max Activating Token Index: 966

Text #17

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.2132. Min Act: -0.1700

Data Index: 8536662 (Open Web Text)

Max Activating Token Index: 899

Text #18

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.1601. Min Act: -0.1700

Data Index: 425427 (Open Web Text)

Max Activating Token Index: 510

Text #19

Max Range: 5.5864. Min Range: -5.5864

Max Act: 5.1768. Min Act: -0.1700

Data Index: 2522474 (Open Web Text)

Max Activating Token Index: 339