Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1034 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.8244. Min Act: -0.1700

Data Index: 4331222 (Open Web Text)

Max Activating Token Index: 472

Text #1

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.7778. Min Act: -0.1700

Data Index: 4630037 (Open Web Text)

Max Activating Token Index: 396

Text #2

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.7404. Min Act: -0.1700

Data Index: 2912497 (Open Web Text)

Max Activating Token Index: 512

Text #3

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.7102. Min Act: -0.1700

Data Index: 892835 (Open Web Text)

Max Activating Token Index: 1008

Text #4

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6908. Min Act: -0.1700

Data Index: 3598288 (Open Web Text)

Max Activating Token Index: 669

Text #5

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6642. Min Act: -0.1700

Data Index: 8617096 (Open Web Text)

Max Activating Token Index: 524

Text #6

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6438. Min Act: -0.1700

Data Index: 15690 (Open Web Text)

Max Activating Token Index: 983

Text #7

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6355. Min Act: -0.1700

Data Index: 6679327 (Open Web Text)

Max Activating Token Index: 632

Text #8

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6294. Min Act: -0.1700

Data Index: 6411448 (Open Web Text)

Max Activating Token Index: 417

Text #9

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6010. Min Act: -0.1700

Data Index: 7176293 (Open Web Text)

Max Activating Token Index: 442

Text #10

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6459. Min Act: -0.1700

Data Index: 81258 (Open Web Text)

Max Activating Token Index: 463

Text #11

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5892. Min Act: -0.1700

Data Index: 7519163 (Open Web Text)

Max Activating Token Index: 621

Text #12

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.6278. Min Act: -0.1700

Data Index: 4230748 (Open Web Text)

Max Activating Token Index: 135

Text #13

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5895. Min Act: -0.1700

Data Index: 6385960 (Open Web Text)

Max Activating Token Index: 300

Text #14

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5559. Min Act: -0.1700

Data Index: 510147 (Open Web Text)

Max Activating Token Index: 797

Text #15

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5730. Min Act: -0.1700

Data Index: 1146466 (Open Web Text)

Max Activating Token Index: 968

Text #16

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5773. Min Act: -0.1700

Data Index: 3428695 (Open Web Text)

Max Activating Token Index: 990

Text #17

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5435. Min Act: -0.1700

Data Index: 1277477 (Open Web Text)

Max Activating Token Index: 82

Text #18

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5718. Min Act: -0.1700

Data Index: 5198168 (Open Web Text)

Max Activating Token Index: 826

Text #19

Max Range: 4.8244. Min Range: -4.8244

Max Act: 4.5374. Min Act: -0.1700

Data Index: 2902391 (Open Web Text)

Max Activating Token Index: 867