Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 145 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.9505. Min Act: -0.1700

Data Index: 721321 (Open Web Text)

Max Activating Token Index: 992

Text #1

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.9554. Min Act: -0.1700

Data Index: 6222852 (Open Web Text)

Max Activating Token Index: 821

Text #2

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.9421. Min Act: -0.1700

Data Index: 836646 (Open Web Text)

Max Activating Token Index: 337

Text #3

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.9012. Min Act: -0.1700

Data Index: 2431583 (Open Web Text)

Max Activating Token Index: 889

Text #4

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.9246. Min Act: -0.1700

Data Index: 255900 (Open Web Text)

Max Activating Token Index: 517

Text #5

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8968. Min Act: -0.1700

Data Index: 2025924 (Open Web Text)

Max Activating Token Index: 1004

Text #6

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.9032. Min Act: -0.1700

Data Index: 7926702 (Open Web Text)

Max Activating Token Index: 546

Text #7

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.9079. Min Act: -0.1700

Data Index: 2528801 (Open Web Text)

Max Activating Token Index: 853

Text #8

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8822. Min Act: -0.1700

Data Index: 2025927 (Open Web Text)

Max Activating Token Index: 353

Text #9

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8872. Min Act: -0.1700

Data Index: 3584450 (Open Web Text)

Max Activating Token Index: 22

Text #10

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8530. Min Act: -0.1700

Data Index: 791717 (Open Web Text)

Max Activating Token Index: 743

Text #11

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8759. Min Act: -0.1700

Data Index: 246835 (Open Web Text)

Max Activating Token Index: 583

Text #12

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8781. Min Act: -0.1700

Data Index: 4029846 (Open Web Text)

Max Activating Token Index: 791

Text #13

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8689. Min Act: -0.1700

Data Index: 3449923 (Open Web Text)

Max Activating Token Index: 317

Text #14

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8840. Min Act: -0.1700

Data Index: 668681 (Open Web Text)

Max Activating Token Index: 433

Text #15

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8527. Min Act: -0.1700

Data Index: 6791421 (Open Web Text)

Max Activating Token Index: 624

Text #16

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8663. Min Act: -0.1700

Data Index: 4251467 (Open Web Text)

Max Activating Token Index: 879

Text #17

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8665. Min Act: -0.1700

Data Index: 5300865 (Open Web Text)

Max Activating Token Index: 526

Text #18

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8617. Min Act: -0.1700

Data Index: 7258887 (Open Web Text)

Max Activating Token Index: 378

Text #19

Max Range: 1.9554. Min Range: -1.9554

Max Act: 1.8560. Min Act: -0.1700

Data Index: 3477646 (Open Web Text)

Max Activating Token Index: 50