Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1014 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.6053. Min Act: -0.1700

Data Index: 7329757 (Open Web Text)

Max Activating Token Index: 817

Text #1

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.5667. Min Act: -0.1700

Data Index: 6131432 (Open Web Text)

Max Activating Token Index: 378

Text #2

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.5626. Min Act: -0.1700

Data Index: 3439083 (Open Web Text)

Max Activating Token Index: 351

Text #3

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.5114. Min Act: -0.1700

Data Index: 3048928 (Open Web Text)

Max Activating Token Index: 636

Text #4

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.4382. Min Act: -0.1700

Data Index: 6574287 (Open Web Text)

Max Activating Token Index: 323

Text #5

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.4560. Min Act: -0.1700

Data Index: 3618699 (Open Web Text)

Max Activating Token Index: 925

Text #6

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.4612. Min Act: -0.1700

Data Index: 4039022 (Open Web Text)

Max Activating Token Index: 665

Text #7

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.4234. Min Act: -0.1700

Data Index: 6462467 (Open Web Text)

Max Activating Token Index: 735

Text #8

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3977. Min Act: -0.1700

Data Index: 678088 (Open Web Text)

Max Activating Token Index: 218

Text #9

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3898. Min Act: -0.1700

Data Index: 2859716 (Open Web Text)

Max Activating Token Index: 206

Text #10

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3786. Min Act: -0.1700

Data Index: 8518396 (Open Web Text)

Max Activating Token Index: 786

Text #11

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.4039. Min Act: -0.1700

Data Index: 6872975 (Open Web Text)

Max Activating Token Index: 519

Text #12

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3650. Min Act: -0.1700

Data Index: 4854894 (Open Web Text)

Max Activating Token Index: 523

Text #13

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3676. Min Act: -0.1700

Data Index: 4999009 (Open Web Text)

Max Activating Token Index: 817

Text #14

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3925. Min Act: -0.1700

Data Index: 599184 (Open Web Text)

Max Activating Token Index: 543

Text #15

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3835. Min Act: -0.1700

Data Index: 4319341 (Open Web Text)

Max Activating Token Index: 165

Text #16

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3332. Min Act: -0.1700

Data Index: 5463485 (Open Web Text)

Max Activating Token Index: 378

Text #17

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3397. Min Act: -0.1700

Data Index: 3782231 (Open Web Text)

Max Activating Token Index: 912

Text #18

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3820. Min Act: -0.1700

Data Index: 2824735 (Open Web Text)

Max Activating Token Index: 417

Text #19

Max Range: 5.6053. Min Range: -5.6053

Max Act: 5.3229. Min Act: -0.1700

Data Index: 6131433 (Open Web Text)

Max Activating Token Index: 504