Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2415 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1756. Min Act: -0.1700

Data Index: 2703194 (Open Web Text)

Max Activating Token Index: 1018

Text #1

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1380. Min Act: -0.1700

Data Index: 1007489 (Open Web Text)

Max Activating Token Index: 47

Text #2

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1277. Min Act: -0.1700

Data Index: 5522176 (Open Web Text)

Max Activating Token Index: 167

Text #3

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1119. Min Act: -0.1700

Data Index: 6426734 (Open Web Text)

Max Activating Token Index: 581

Text #4

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1240. Min Act: -0.1700

Data Index: 7074430 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1231. Min Act: -0.1700

Data Index: 8219091 (Open Web Text)

Max Activating Token Index: 733

Text #6

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1008. Min Act: -0.1700

Data Index: 2195467 (Open Web Text)

Max Activating Token Index: 134

Text #7

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1044. Min Act: -0.1700

Data Index: 1499623 (Open Web Text)

Max Activating Token Index: 236

Text #8

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.0903. Min Act: -0.1700

Data Index: 8334444 (Open Web Text)

Max Activating Token Index: 771

Text #9

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1056. Min Act: -0.1700

Data Index: 761230 (Open Web Text)

Max Activating Token Index: 737

Text #10

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1007. Min Act: -0.1700

Data Index: 2407588 (Open Web Text)

Max Activating Token Index: 264

Text #11

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1012. Min Act: -0.1700

Data Index: 2377520 (Open Web Text)

Max Activating Token Index: 539

Text #12

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1097. Min Act: -0.1700

Data Index: 2365418 (Open Web Text)

Max Activating Token Index: 190

Text #13

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.0977. Min Act: -0.1700

Data Index: 3566828 (Open Web Text)

Max Activating Token Index: 536

Text #14

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1098. Min Act: -0.1700

Data Index: 1347837 (Open Web Text)

Max Activating Token Index: 700

Text #15

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.0847. Min Act: -0.1700

Data Index: 6128064 (Open Web Text)

Max Activating Token Index: 171

Text #16

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.1089. Min Act: -0.1700

Data Index: 4969293 (Open Web Text)

Max Activating Token Index: 617

Text #17

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.0787. Min Act: -0.1700

Data Index: 1019146 (Open Web Text)

Max Activating Token Index: 477

Text #18

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.0824. Min Act: -0.1700

Data Index: 3064433 (Open Web Text)

Max Activating Token Index: 713

Text #19

Max Range: 2.1756. Min Range: -2.1756

Max Act: 2.0964. Min Act: -0.1700

Data Index: 3940023 (Open Web Text)

Max Activating Token Index: 228