Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2191 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.6747. Min Act: -0.1700

Data Index: 159166 (Open Web Text)

Max Activating Token Index: 707

Text #1

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5934. Min Act: -0.1700

Data Index: 5473140 (Open Web Text)

Max Activating Token Index: 829

Text #2

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5976. Min Act: -0.1700

Data Index: 4944061 (Open Web Text)

Max Activating Token Index: 672

Text #3

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5575. Min Act: -0.1700

Data Index: 7063414 (Open Web Text)

Max Activating Token Index: 910

Text #4

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5497. Min Act: -0.1700

Data Index: 1623423 (Open Web Text)

Max Activating Token Index: 230

Text #5

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5781. Min Act: -0.1700

Data Index: 3379596 (Open Web Text)

Max Activating Token Index: 576

Text #6

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5722. Min Act: -0.1700

Data Index: 4890667 (Open Web Text)

Max Activating Token Index: 531

Text #7

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5012. Min Act: -0.1700

Data Index: 4037728 (Open Web Text)

Max Activating Token Index: 133

Text #8

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4974. Min Act: -0.1700

Data Index: 2501655 (Open Web Text)

Max Activating Token Index: 533

Text #9

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5052. Min Act: -0.1700

Data Index: 2553364 (Open Web Text)

Max Activating Token Index: 669

Text #10

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4956. Min Act: -0.1700

Data Index: 5611859 (Open Web Text)

Max Activating Token Index: 376

Text #11

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5004. Min Act: -0.1700

Data Index: 5754925 (Open Web Text)

Max Activating Token Index: 474

Text #12

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4859. Min Act: -0.1700

Data Index: 3406323 (Open Web Text)

Max Activating Token Index: 683

Text #13

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.5157. Min Act: -0.1700

Data Index: 273849 (Open Web Text)

Max Activating Token Index: 674

Text #14

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4719. Min Act: -0.1700

Data Index: 8114555 (Open Web Text)

Max Activating Token Index: 600

Text #15

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4588. Min Act: -0.1700

Data Index: 7104000 (Open Web Text)

Max Activating Token Index: 470

Text #16

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4612. Min Act: -0.1700

Data Index: 3197019 (Open Web Text)

Max Activating Token Index: 914

Text #17

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4530. Min Act: -0.1700

Data Index: 6257192 (Open Web Text)

Max Activating Token Index: 991

Text #18

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4829. Min Act: -0.1700

Data Index: 1324489 (Open Web Text)

Max Activating Token Index: 802

Text #19

Max Range: 4.6747. Min Range: -4.6747

Max Act: 4.4583. Min Act: -0.1700

Data Index: 4760347 (Open Web Text)

Max Activating Token Index: 224