Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 4 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1264. Min Range: -3.1264

Max Act: 3.1264. Min Act: -0.1700

Data Index: 7947781 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 3.1264. Min Range: -3.1264

Max Act: 3.0448. Min Act: -0.1700

Data Index: 2986581 (Open Web Text)

Max Activating Token Index: 482

Text #2

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8950. Min Act: -0.1700

Data Index: 826523 (Open Web Text)

Max Activating Token Index: 857

Text #3

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.9023. Min Act: -0.1700

Data Index: 6929305 (Open Web Text)

Max Activating Token Index: 357

Text #4

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8314. Min Act: -0.1700

Data Index: 3386190 (Open Web Text)

Max Activating Token Index: 169

Text #5

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8362. Min Act: -0.1700

Data Index: 7518761 (Open Web Text)

Max Activating Token Index: 555

Text #6

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8410. Min Act: -0.1700

Data Index: 7132644 (Open Web Text)

Max Activating Token Index: 119

Text #7

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8309. Min Act: -0.1700

Data Index: 6108452 (Open Web Text)

Max Activating Token Index: 846

Text #8

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8122. Min Act: -0.1700

Data Index: 3026131 (Open Web Text)

Max Activating Token Index: 179

Text #9

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8306. Min Act: -0.1700

Data Index: 519931 (Open Web Text)

Max Activating Token Index: 255

Text #10

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8168. Min Act: -0.1700

Data Index: 5500163 (Open Web Text)

Max Activating Token Index: 992

Text #11

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8167. Min Act: -0.1700

Data Index: 8619577 (Open Web Text)

Max Activating Token Index: 267

Text #12

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8156. Min Act: -0.1700

Data Index: 6142085 (Open Web Text)

Max Activating Token Index: 688

Text #13

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.8079. Min Act: -0.1700

Data Index: 503885 (Open Web Text)

Max Activating Token Index: 110

Text #14

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.7751. Min Act: -0.1700

Data Index: 1877615 (Open Web Text)

Max Activating Token Index: 484

Text #15

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.7549. Min Act: -0.1700

Data Index: 3204480 (Open Web Text)

Max Activating Token Index: 517

Text #16

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.7526. Min Act: -0.1700

Data Index: 405784 (Open Web Text)

Max Activating Token Index: 628

Text #17

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.7675. Min Act: -0.1700

Data Index: 5220733 (Open Web Text)

Max Activating Token Index: 693

Text #18

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.7728. Min Act: -0.1700

Data Index: 3023284 (Open Web Text)

Max Activating Token Index: 402

Text #19

Max Range: 3.1264. Min Range: -3.1264

Max Act: 2.7577. Min Act: -0.1700

Data Index: 5658019 (Open Web Text)

Max Activating Token Index: 342