Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1170 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.2384. Min Act: -0.1700

Data Index: 2429510 (Open Web Text)

Max Activating Token Index: 217

Text #1

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.2187. Min Act: -0.1700

Data Index: 5691308 (Open Web Text)

Max Activating Token Index: 805

Text #2

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.2099. Min Act: -0.1700

Data Index: 5326456 (Open Web Text)

Max Activating Token Index: 599

Text #3

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1608. Min Act: -0.1700

Data Index: 1239192 (Open Web Text)

Max Activating Token Index: 503

Text #4

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1453. Min Act: -0.1700

Data Index: 5615642 (Open Web Text)

Max Activating Token Index: 108

Text #5

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1262. Min Act: -0.1700

Data Index: 4192101 (Open Web Text)

Max Activating Token Index: 146

Text #6

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1195. Min Act: -0.1700

Data Index: 1586089 (Open Web Text)

Max Activating Token Index: 558

Text #7

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1220. Min Act: -0.1700

Data Index: 3807045 (Open Web Text)

Max Activating Token Index: 310

Text #8

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1217. Min Act: -0.1700

Data Index: 190323 (Open Web Text)

Max Activating Token Index: 295

Text #9

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1216. Min Act: -0.1700

Data Index: 2709989 (Open Web Text)

Max Activating Token Index: 187

Text #10

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1192. Min Act: -0.1700

Data Index: 570764 (Open Web Text)

Max Activating Token Index: 496

Text #11

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1110. Min Act: -0.1700

Data Index: 7135041 (Open Web Text)

Max Activating Token Index: 278

Text #12

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1037. Min Act: -0.1700

Data Index: 2693742 (Open Web Text)

Max Activating Token Index: 864

Text #13

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1220. Min Act: -0.1700

Data Index: 566463 (Open Web Text)

Max Activating Token Index: 315

Text #14

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.0929. Min Act: -0.1700

Data Index: 3699151 (Open Web Text)

Max Activating Token Index: 686

Text #15

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1034. Min Act: -0.1700

Data Index: 5219957 (Open Web Text)

Max Activating Token Index: 276

Text #16

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.0839. Min Act: -0.1700

Data Index: 631362 (Open Web Text)

Max Activating Token Index: 855

Text #17

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.0992. Min Act: -0.1700

Data Index: 75851 (Open Web Text)

Max Activating Token Index: 601

Text #18

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.0881. Min Act: -0.1700

Data Index: 2726719 (Open Web Text)

Max Activating Token Index: 215

Text #19

Max Range: 3.2384. Min Range: -3.2384

Max Act: 3.1026. Min Act: -0.1700

Data Index: 1256691 (Open Web Text)

Max Activating Token Index: 129