Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2957 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.6563. Min Act: -0.1700

Data Index: 7155143 (Open Web Text)

Max Activating Token Index: 10

Text #1

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.5596. Min Act: -0.1700

Data Index: 1484999 (Open Web Text)

Max Activating Token Index: 938

Text #2

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.5512. Min Act: -0.1700

Data Index: 2163310 (Open Web Text)

Max Activating Token Index: 37

Text #3

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.5326. Min Act: -0.1700

Data Index: 7948509 (Open Web Text)

Max Activating Token Index: 58

Text #4

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.5159. Min Act: -0.1700

Data Index: 7899952 (Open Web Text)

Max Activating Token Index: 80

Text #5

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.4971. Min Act: -0.1700

Data Index: 6878299 (Open Web Text)

Max Activating Token Index: 56

Text #6

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.4304. Min Act: -0.1700

Data Index: 585703 (Open Web Text)

Max Activating Token Index: 82

Text #7

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.4023. Min Act: -0.1700

Data Index: 342119 (Open Web Text)

Max Activating Token Index: 11

Text #8

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3960. Min Act: -0.1700

Data Index: 2700086 (Open Web Text)

Max Activating Token Index: 112

Text #9

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.4539. Min Act: -0.1700

Data Index: 3174673 (Open Web Text)

Max Activating Token Index: 589

Text #10

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.4116. Min Act: -0.1700

Data Index: 4484585 (Open Web Text)

Max Activating Token Index: 133

Text #11

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3711. Min Act: -0.1700

Data Index: 619755 (Open Web Text)

Max Activating Token Index: 579

Text #12

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3704. Min Act: -0.1700

Data Index: 4548724 (Open Web Text)

Max Activating Token Index: 99

Text #13

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3388. Min Act: -0.1700

Data Index: 3841337 (Open Web Text)

Max Activating Token Index: 518

Text #14

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3499. Min Act: -0.1700

Data Index: 6376963 (Open Web Text)

Max Activating Token Index: 20

Text #15

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3151. Min Act: -0.1700

Data Index: 2778571 (Open Web Text)

Max Activating Token Index: 700

Text #16

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3068. Min Act: -0.1700

Data Index: 5798873 (Open Web Text)

Max Activating Token Index: 168

Text #17

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.2973. Min Act: -0.1700

Data Index: 4842996 (Open Web Text)

Max Activating Token Index: 97

Text #18

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3267. Min Act: -0.1700

Data Index: 7899951 (Open Web Text)

Max Activating Token Index: 199

Text #19

Max Range: 3.6563. Min Range: -3.6563

Max Act: 3.3108. Min Act: -0.1700

Data Index: 2106135 (Open Web Text)

Max Activating Token Index: 28