Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 179 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.8616. Min Act: -0.1700

Data Index: 989182 (Open Web Text)

Max Activating Token Index: 497

Text #1

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.8299. Min Act: -0.1700

Data Index: 272621 (Open Web Text)

Max Activating Token Index: 485

Text #2

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.8258. Min Act: -0.1700

Data Index: 8256824 (Open Web Text)

Max Activating Token Index: 919

Text #3

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.8266. Min Act: -0.1700

Data Index: 4010380 (Open Web Text)

Max Activating Token Index: 445

Text #4

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.8019. Min Act: -0.1700

Data Index: 172410 (Open Web Text)

Max Activating Token Index: 374

Text #5

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.8093. Min Act: -0.1700

Data Index: 4916245 (Open Web Text)

Max Activating Token Index: 505

Text #6

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7984. Min Act: -0.1700

Data Index: 5574211 (Open Web Text)

Max Activating Token Index: 471

Text #7

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7981. Min Act: -0.1700

Data Index: 7307272 (Open Web Text)

Max Activating Token Index: 410

Text #8

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7902. Min Act: -0.1700

Data Index: 7778741 (Open Web Text)

Max Activating Token Index: 544

Text #9

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7904. Min Act: -0.1700

Data Index: 2340377 (Open Web Text)

Max Activating Token Index: 413

Text #10

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.8143. Min Act: -0.1700

Data Index: 1019373 (Open Web Text)

Max Activating Token Index: 907

Text #11

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7886. Min Act: -0.1700

Data Index: 4084610 (Open Web Text)

Max Activating Token Index: 248

Text #12

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7627. Min Act: -0.1700

Data Index: 4194556 (Open Web Text)

Max Activating Token Index: 515

Text #13

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7713. Min Act: -0.1700

Data Index: 1704616 (Open Web Text)

Max Activating Token Index: 281

Text #14

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7561. Min Act: -0.1700

Data Index: 4091134 (Open Web Text)

Max Activating Token Index: 743

Text #15

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7536. Min Act: -0.1700

Data Index: 3821462 (Open Web Text)

Max Activating Token Index: 729

Text #16

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7559. Min Act: -0.1700

Data Index: 3347750 (Open Web Text)

Max Activating Token Index: 408

Text #17

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7617. Min Act: -0.1700

Data Index: 940973 (Open Web Text)

Max Activating Token Index: 482

Text #18

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7759. Min Act: -0.1700

Data Index: 3240543 (Open Web Text)

Max Activating Token Index: 697

Text #19

Max Range: 3.8616. Min Range: -3.8616

Max Act: 3.7506. Min Act: -0.1700

Data Index: 3198799 (Open Web Text)

Max Activating Token Index: 941