Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 897 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.3143. Min Act: -0.1700

Data Index: 325408 (Open Web Text)

Max Activating Token Index: 430

Text #1

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.1740. Min Act: -0.1700

Data Index: 1316509 (Open Web Text)

Max Activating Token Index: 795

Text #2

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.1541. Min Act: -0.1700

Data Index: 3828710 (Open Web Text)

Max Activating Token Index: 206

Text #3

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.1230. Min Act: -0.1700

Data Index: 4181331 (Open Web Text)

Max Activating Token Index: 454

Text #4

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.1544. Min Act: -0.1700

Data Index: 8507693 (Open Web Text)

Max Activating Token Index: 853

Text #5

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.1318. Min Act: -0.1700

Data Index: 5853697 (Open Web Text)

Max Activating Token Index: 326

Text #6

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0853. Min Act: -0.1700

Data Index: 3918268 (Open Web Text)

Max Activating Token Index: 748

Text #7

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0773. Min Act: -0.1700

Data Index: 5548565 (Open Web Text)

Max Activating Token Index: 69

Text #8

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.1087. Min Act: -0.1700

Data Index: 1199154 (Open Web Text)

Max Activating Token Index: 417

Text #9

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0959. Min Act: -0.1700

Data Index: 2318416 (Open Web Text)

Max Activating Token Index: 910

Text #10

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0968. Min Act: -0.1700

Data Index: 5261811 (Open Web Text)

Max Activating Token Index: 184

Text #11

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0720. Min Act: -0.1700

Data Index: 4290343 (Open Web Text)

Max Activating Token Index: 105

Text #12

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0614. Min Act: -0.1700

Data Index: 5509143 (Open Web Text)

Max Activating Token Index: 96

Text #13

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0450. Min Act: -0.1700

Data Index: 2406839 (Open Web Text)

Max Activating Token Index: 429

Text #14

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0486. Min Act: -0.1700

Data Index: 7878460 (Open Web Text)

Max Activating Token Index: 261

Text #15

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0422. Min Act: -0.1700

Data Index: 4100952 (Open Web Text)

Max Activating Token Index: 134

Text #16

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0028. Min Act: -0.1700

Data Index: 613074 (Open Web Text)

Max Activating Token Index: 869

Text #17

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0127. Min Act: -0.1700

Data Index: 839611 (Open Web Text)

Max Activating Token Index: 177

Text #18

Max Range: 4.3143. Min Range: -4.3143

Max Act: 4.0497. Min Act: -0.1700

Data Index: 6831358 (Open Web Text)

Max Activating Token Index: 544

Text #19

Max Range: 4.3143. Min Range: -4.3143

Max Act: 3.9908. Min Act: -0.1700

Data Index: 8658962 (Open Web Text)

Max Activating Token Index: 1008