Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 409 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.2852. Min Act: -0.1700

Data Index: 1090086 (Open Web Text)

Max Activating Token Index: 491

Text #1

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.1670. Min Act: -0.1700

Data Index: 1101310 (Open Web Text)

Max Activating Token Index: 50

Text #2

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.1838. Min Act: -0.1700

Data Index: 2220243 (Open Web Text)

Max Activating Token Index: 667

Text #3

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.1551. Min Act: -0.1700

Data Index: 2583811 (Open Web Text)

Max Activating Token Index: 686

Text #4

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0817. Min Act: -0.1700

Data Index: 496234 (Open Web Text)

Max Activating Token Index: 365

Text #5

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0397. Min Act: -0.1700

Data Index: 480724 (Open Web Text)

Max Activating Token Index: 182

Text #6

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0650. Min Act: -0.1700

Data Index: 2968652 (Open Web Text)

Max Activating Token Index: 802

Text #7

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0536. Min Act: -0.1700

Data Index: 3610164 (Open Web Text)

Max Activating Token Index: 376

Text #8

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0462. Min Act: -0.1700

Data Index: 5454087 (Open Web Text)

Max Activating Token Index: 19

Text #9

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0434. Min Act: -0.1700

Data Index: 4597999 (Open Web Text)

Max Activating Token Index: 912

Text #10

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0035. Min Act: -0.1700

Data Index: 8336224 (Open Web Text)

Max Activating Token Index: 719

Text #11

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0445. Min Act: -0.1700

Data Index: 5964502 (Open Web Text)

Max Activating Token Index: 324

Text #12

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0261. Min Act: -0.1700

Data Index: 8242057 (Open Web Text)

Max Activating Token Index: 645

Text #13

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0331. Min Act: -0.1700

Data Index: 8143859 (Open Web Text)

Max Activating Token Index: 650

Text #14

Max Range: 4.2852. Min Range: -4.2852

Max Act: 4.0285. Min Act: -0.1700

Data Index: 2560951 (Open Web Text)

Max Activating Token Index: 940

Text #15

Max Range: 4.2852. Min Range: -4.2852

Max Act: 3.9768. Min Act: -0.1700

Data Index: 4083745 (Open Web Text)

Max Activating Token Index: 229

Text #16

Max Range: 4.2852. Min Range: -4.2852

Max Act: 3.9435. Min Act: -0.1700

Data Index: 3074711 (Open Web Text)

Max Activating Token Index: 596

Text #17

Max Range: 4.2852. Min Range: -4.2852

Max Act: 3.9722. Min Act: -0.1700

Data Index: 321339 (Open Web Text)

Max Activating Token Index: 100

Text #18

Max Range: 4.2852. Min Range: -4.2852

Max Act: 3.9688. Min Act: -0.1700

Data Index: 7021457 (Open Web Text)

Max Activating Token Index: 401

Text #19

Max Range: 4.2852. Min Range: -4.2852

Max Act: 3.9660. Min Act: -0.1700

Data Index: 1013706 (Open Web Text)

Max Activating Token Index: 759