Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 267 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.5439. Min Act: -0.1501

Data Index: 5103797 (Open Web Text)

Max Activating Token Index: 84

Text #1

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.4998. Min Act: -0.1696

Data Index: 1667203 (Open Web Text)

Max Activating Token Index: 325

Text #2

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.4465. Min Act: -0.0947

Data Index: 8090643 (Open Web Text)

Max Activating Token Index: 346

Text #3

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.4124. Min Act: -0.0947

Data Index: 8347676 (Open Web Text)

Max Activating Token Index: 214

Text #4

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.4126. Min Act: -0.1021

Data Index: 3189028 (Open Web Text)

Max Activating Token Index: 384

Text #5

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.4081. Min Act: -0.1136

Data Index: 561933 (Open Web Text)

Max Activating Token Index: 424

Text #6

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.4012. Min Act: -0.1513

Data Index: 2038768 (Open Web Text)

Max Activating Token Index: 14

Text #7

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3998. Min Act: -0.1501

Data Index: 7095523 (Open Web Text)

Max Activating Token Index: 939

Text #8

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.4019. Min Act: -0.1379

Data Index: 378163 (Open Web Text)

Max Activating Token Index: 281

Text #9

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3898. Min Act: -0.1019

Data Index: 3475674 (Open Web Text)

Max Activating Token Index: 1005

Text #10

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3806. Min Act: -0.1677

Data Index: 7367140 (Open Web Text)

Max Activating Token Index: 889

Text #11

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3725. Min Act: -0.1311

Data Index: 2712046 (Open Web Text)

Max Activating Token Index: 963

Text #12

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3612. Min Act: -0.1366

Data Index: 3789330 (Open Web Text)

Max Activating Token Index: 48

Text #13

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3661. Min Act: -0.1437

Data Index: 2094128 (Open Web Text)

Max Activating Token Index: 570

Text #14

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3659. Min Act: -0.1687

Data Index: 7617806 (Open Web Text)

Max Activating Token Index: 472

Text #15

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3718. Min Act: -0.1444

Data Index: 8347677 (Open Web Text)

Max Activating Token Index: 450

Text #16

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3589. Min Act: -0.1541

Data Index: 1028234 (Open Web Text)

Max Activating Token Index: 139

Text #17

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3657. Min Act: -0.0978

Data Index: 5223493 (Open Web Text)

Max Activating Token Index: 19

Text #18

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3516. Min Act: -0.1447

Data Index: 7358771 (Open Web Text)

Max Activating Token Index: 1007

Text #19

Max Range: 1.5439. Min Range: -1.5439

Max Act: 1.3566. Min Act: -0.1244

Data Index: 4674231 (Open Web Text)

Max Activating Token Index: 916