Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 393 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.8096. Min Act: -0.1683

Data Index: 2606537 (Open Web Text)

Max Activating Token Index: 73

Text #1

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.7925. Min Act: -0.1700

Data Index: 498347 (Open Web Text)

Max Activating Token Index: 539

Text #2

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.7439. Min Act: -0.1700

Data Index: 3579792 (Open Web Text)

Max Activating Token Index: 692

Text #3

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.7151. Min Act: -0.1700

Data Index: 4584007 (Open Web Text)

Max Activating Token Index: 422

Text #4

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.7051. Min Act: -0.1700

Data Index: 1531066 (Open Web Text)

Max Activating Token Index: 990

Text #5

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.7012. Min Act: -0.1700

Data Index: 7816682 (Open Web Text)

Max Activating Token Index: 493

Text #6

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6997. Min Act: -0.1700

Data Index: 334246 (Open Web Text)

Max Activating Token Index: 409

Text #7

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6898. Min Act: -0.1700

Data Index: 7266313 (Open Web Text)

Max Activating Token Index: 212

Text #8

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6911. Min Act: -0.1700

Data Index: 200824 (Open Web Text)

Max Activating Token Index: 683

Text #9

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6782. Min Act: -0.1700

Data Index: 194088 (Open Web Text)

Max Activating Token Index: 511

Text #10

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6844. Min Act: -0.1699

Data Index: 4883733 (Open Web Text)

Max Activating Token Index: 182

Text #11

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6773. Min Act: -0.1700

Data Index: 4471473 (Open Web Text)

Max Activating Token Index: 413

Text #12

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6779. Min Act: -0.1700

Data Index: 2365474 (Open Web Text)

Max Activating Token Index: 759

Text #13

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6666. Min Act: -0.1691

Data Index: 7251163 (Open Web Text)

Max Activating Token Index: 464

Text #14

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6675. Min Act: -0.1700

Data Index: 7398509 (Open Web Text)

Max Activating Token Index: 107

Text #15

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6653. Min Act: -0.1700

Data Index: 4621759 (Open Web Text)

Max Activating Token Index: 143

Text #16

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6747. Min Act: -0.1698

Data Index: 2210571 (Open Web Text)

Max Activating Token Index: 230

Text #17

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6519. Min Act: -0.1700

Data Index: 3452257 (Open Web Text)

Max Activating Token Index: 627

Text #18

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6573. Min Act: -0.1700

Data Index: 3504448 (Open Web Text)

Max Activating Token Index: 131

Text #19

Max Range: 1.8096. Min Range: -1.8096

Max Act: 1.6630. Min Act: -0.1700

Data Index: 3005629 (Open Web Text)

Max Activating Token Index: 316