Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 187 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.9058. Min Act: -0.1700

Data Index: 5777968 (Open Web Text)

Max Activating Token Index: 289

Text #1

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.8229. Min Act: -0.1700

Data Index: 1364189 (Open Web Text)

Max Activating Token Index: 538

Text #2

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.8562. Min Act: -0.1700

Data Index: 5503860 (Open Web Text)

Max Activating Token Index: 957

Text #3

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.8095. Min Act: -0.1700

Data Index: 4397026 (Open Web Text)

Max Activating Token Index: 869

Text #4

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7885. Min Act: -0.1700

Data Index: 4819731 (Open Web Text)

Max Activating Token Index: 141

Text #5

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.8044. Min Act: -0.1700

Data Index: 5095850 (Open Web Text)

Max Activating Token Index: 660

Text #6

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7583. Min Act: -0.1700

Data Index: 4625788 (Open Web Text)

Max Activating Token Index: 407

Text #7

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7878. Min Act: -0.1700

Data Index: 6735619 (Open Web Text)

Max Activating Token Index: 848

Text #8

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7704. Min Act: -0.1700

Data Index: 5000917 (Open Web Text)

Max Activating Token Index: 461

Text #9

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7037. Min Act: -0.1700

Data Index: 6163980 (Open Web Text)

Max Activating Token Index: 858

Text #10

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7491. Min Act: -0.1700

Data Index: 4365796 (Open Web Text)

Max Activating Token Index: 548

Text #11

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7234. Min Act: -0.1700

Data Index: 5923890 (Open Web Text)

Max Activating Token Index: 608

Text #12

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7054. Min Act: -0.1700

Data Index: 8230052 (Open Web Text)

Max Activating Token Index: 1014

Text #13

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7477. Min Act: -0.1700

Data Index: 4500039 (Open Web Text)

Max Activating Token Index: 786

Text #14

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7321. Min Act: -0.1700

Data Index: 8595174 (Open Web Text)

Max Activating Token Index: 203

Text #15

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7057. Min Act: -0.1700

Data Index: 8145698 (Open Web Text)

Max Activating Token Index: 974

Text #16

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.6875. Min Act: -0.1700

Data Index: 2743249 (Open Web Text)

Max Activating Token Index: 893

Text #17

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.7404. Min Act: -0.1700

Data Index: 485688 (Open Web Text)

Max Activating Token Index: 631

Text #18

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.6977. Min Act: -0.1700

Data Index: 2464386 (Open Web Text)

Max Activating Token Index: 298

Text #19

Max Range: 3.9058. Min Range: -3.9058

Max Act: 3.6787. Min Act: -0.1700

Data Index: 5956717 (Open Web Text)

Max Activating Token Index: 69