Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 964 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1128. Min Range: -4.1128

Max Act: 4.1128. Min Act: -0.1700

Data Index: 1726017 (Open Web Text)

Max Activating Token Index: 185

Text #1

Max Range: 4.1128. Min Range: -4.1128

Max Act: 4.0253. Min Act: -0.1700

Data Index: 18150 (Open Web Text)

Max Activating Token Index: 17

Text #2

Max Range: 4.1128. Min Range: -4.1128

Max Act: 4.0494. Min Act: -0.1700

Data Index: 1938397 (Open Web Text)

Max Activating Token Index: 447

Text #3

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.9687. Min Act: -0.1700

Data Index: 2716240 (Open Web Text)

Max Activating Token Index: 42

Text #4

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.9918. Min Act: -0.1700

Data Index: 6120357 (Open Web Text)

Max Activating Token Index: 919

Text #5

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.8980. Min Act: -0.1700

Data Index: 8204078 (Open Web Text)

Max Activating Token Index: 121

Text #6

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.8660. Min Act: -0.1700

Data Index: 5336765 (Open Web Text)

Max Activating Token Index: 698

Text #7

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.8377. Min Act: -0.1700

Data Index: 8461532 (Open Web Text)

Max Activating Token Index: 55

Text #8

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.8324. Min Act: -0.1699

Data Index: 429703 (Open Web Text)

Max Activating Token Index: 334

Text #9

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.8337. Min Act: -0.1700

Data Index: 8413044 (Open Web Text)

Max Activating Token Index: 935

Text #10

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7915. Min Act: -0.1700

Data Index: 8604242 (Open Web Text)

Max Activating Token Index: 1015

Text #11

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.8429. Min Act: -0.1700

Data Index: 1592132 (Open Web Text)

Max Activating Token Index: 477

Text #12

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.8154. Min Act: -0.1700

Data Index: 7472712 (Open Web Text)

Max Activating Token Index: 533

Text #13

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7832. Min Act: -0.1700

Data Index: 1847836 (Open Web Text)

Max Activating Token Index: 587

Text #14

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7976. Min Act: -0.1700

Data Index: 4999475 (Open Web Text)

Max Activating Token Index: 948

Text #15

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7806. Min Act: -0.1700

Data Index: 5772075 (Open Web Text)

Max Activating Token Index: 905

Text #16

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7678. Min Act: -0.1700

Data Index: 4128005 (Open Web Text)

Max Activating Token Index: 99

Text #17

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7740. Min Act: -0.1700

Data Index: 6231887 (Open Web Text)

Max Activating Token Index: 976

Text #18

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7846. Min Act: -0.1700

Data Index: 318443 (Open Web Text)

Max Activating Token Index: 52

Text #19

Max Range: 4.1128. Min Range: -4.1128

Max Act: 3.7857. Min Act: -0.1700

Data Index: 390285 (Open Web Text)

Max Activating Token Index: 169