Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1073 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.4597. Min Act: -0.1700

Data Index: 8076667 (Open Web Text)

Max Activating Token Index: 260

Text #1

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.4091. Min Act: -0.1700

Data Index: 2929994 (Open Web Text)

Max Activating Token Index: 864

Text #2

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.3951. Min Act: -0.1700

Data Index: 7981784 (Open Web Text)

Max Activating Token Index: 732

Text #3

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.3431. Min Act: -0.1700

Data Index: 7227604 (Open Web Text)

Max Activating Token Index: 522

Text #4

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.3775. Min Act: -0.1700

Data Index: 7106054 (Open Web Text)

Max Activating Token Index: 759

Text #5

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.3212. Min Act: -0.1700

Data Index: 7412008 (Open Web Text)

Max Activating Token Index: 361

Text #6

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.3132. Min Act: -0.1700

Data Index: 4658574 (Open Web Text)

Max Activating Token Index: 11

Text #7

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.3154. Min Act: -0.1700

Data Index: 4697130 (Open Web Text)

Max Activating Token Index: 286

Text #8

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2903. Min Act: -0.1700

Data Index: 78608 (Open Web Text)

Max Activating Token Index: 693

Text #9

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2790. Min Act: -0.1700

Data Index: 3895985 (Open Web Text)

Max Activating Token Index: 377

Text #10

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2699. Min Act: -0.1700

Data Index: 5843829 (Open Web Text)

Max Activating Token Index: 705

Text #11

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2646. Min Act: -0.1700

Data Index: 2831195 (Open Web Text)

Max Activating Token Index: 827

Text #12

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2955. Min Act: -0.1700

Data Index: 6143472 (Open Web Text)

Max Activating Token Index: 1020

Text #13

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.3221. Min Act: -0.1700

Data Index: 6059493 (Open Web Text)

Max Activating Token Index: 166

Text #14

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2494. Min Act: -0.1700

Data Index: 4765750 (Open Web Text)

Max Activating Token Index: 865

Text #15

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2857. Min Act: -0.1700

Data Index: 5519122 (Open Web Text)

Max Activating Token Index: 889

Text #16

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2569. Min Act: -0.1700

Data Index: 5303784 (Open Web Text)

Max Activating Token Index: 685

Text #17

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2450. Min Act: -0.1700

Data Index: 4420911 (Open Web Text)

Max Activating Token Index: 964

Text #18

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2784. Min Act: -0.1700

Data Index: 6414784 (Open Web Text)

Max Activating Token Index: 735

Text #19

Max Range: 4.4597. Min Range: -4.4597

Max Act: 4.2423. Min Act: -0.1700

Data Index: 989832 (Open Web Text)

Max Activating Token Index: 481