Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1324 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4669. Min Act: -0.1700

Data Index: 6587891 (Open Web Text)

Max Activating Token Index: 441

Text #1

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4839. Min Act: -0.1700

Data Index: 3455550 (Open Web Text)

Max Activating Token Index: 522

Text #2

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4641. Min Act: -0.1700

Data Index: 252574 (Open Web Text)

Max Activating Token Index: 937

Text #3

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4297. Min Act: -0.1700

Data Index: 962560 (Open Web Text)

Max Activating Token Index: 298

Text #4

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4233. Min Act: -0.1700

Data Index: 5723162 (Open Web Text)

Max Activating Token Index: 93

Text #5

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4362. Min Act: -0.1700

Data Index: 3132815 (Open Web Text)

Max Activating Token Index: 746

Text #6

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4315. Min Act: -0.1700

Data Index: 4752725 (Open Web Text)

Max Activating Token Index: 417

Text #7

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4614. Min Act: -0.1700

Data Index: 3526331 (Open Web Text)

Max Activating Token Index: 403

Text #8

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4331. Min Act: -0.1700

Data Index: 4102315 (Open Web Text)

Max Activating Token Index: 952

Text #9

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4210. Min Act: -0.1700

Data Index: 1614079 (Open Web Text)

Max Activating Token Index: 992

Text #10

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4011. Min Act: -0.1700

Data Index: 4976405 (Open Web Text)

Max Activating Token Index: 577

Text #11

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4057. Min Act: -0.1700

Data Index: 5281748 (Open Web Text)

Max Activating Token Index: 751

Text #12

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4006. Min Act: -0.1700

Data Index: 2839918 (Open Web Text)

Max Activating Token Index: 526

Text #13

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4119. Min Act: -0.1700

Data Index: 6585489 (Open Web Text)

Max Activating Token Index: 928

Text #14

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.4057. Min Act: -0.1700

Data Index: 1211857 (Open Web Text)

Max Activating Token Index: 491

Text #15

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.3886. Min Act: -0.1700

Data Index: 3374038 (Open Web Text)

Max Activating Token Index: 180

Text #16

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.3820. Min Act: -0.1700

Data Index: 118395 (Open Web Text)

Max Activating Token Index: 207

Text #17

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.3889. Min Act: -0.1700

Data Index: 4187363 (Open Web Text)

Max Activating Token Index: 185

Text #18

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.3792. Min Act: -0.1700

Data Index: 2247123 (Open Web Text)

Max Activating Token Index: 71

Text #19

Max Range: 3.4839. Min Range: -3.4839

Max Act: 3.3691. Min Act: -0.1700

Data Index: 412692 (Open Web Text)

Max Activating Token Index: 958