Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2157 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9553. Min Act: -0.1700

Data Index: 3494425 (Open Web Text)

Max Activating Token Index: 243

Text #1

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9429. Min Act: -0.1700

Data Index: 6908910 (Open Web Text)

Max Activating Token Index: 111

Text #2

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9453. Min Act: -0.1700

Data Index: 2869180 (Open Web Text)

Max Activating Token Index: 304

Text #3

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9411. Min Act: -0.1700

Data Index: 901026 (Open Web Text)

Max Activating Token Index: 631

Text #4

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9201. Min Act: -0.1700

Data Index: 1656357 (Open Web Text)

Max Activating Token Index: 735

Text #5

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9081. Min Act: -0.1700

Data Index: 2267091 (Open Web Text)

Max Activating Token Index: 48

Text #6

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9040. Min Act: -0.1700

Data Index: 6534773 (Open Web Text)

Max Activating Token Index: 591

Text #7

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9051. Min Act: -0.1700

Data Index: 6424533 (Open Web Text)

Max Activating Token Index: 461

Text #8

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.9179. Min Act: -0.1700

Data Index: 5754581 (Open Web Text)

Max Activating Token Index: 64

Text #9

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8956. Min Act: -0.1700

Data Index: 3497726 (Open Web Text)

Max Activating Token Index: 570

Text #10

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8914. Min Act: -0.1700

Data Index: 2950843 (Open Web Text)

Max Activating Token Index: 747

Text #11

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8816. Min Act: -0.1700

Data Index: 2002531 (Open Web Text)

Max Activating Token Index: 480

Text #12

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8822. Min Act: -0.1700

Data Index: 6643196 (Open Web Text)

Max Activating Token Index: 215

Text #13

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8982. Min Act: -0.1699

Data Index: 993857 (Open Web Text)

Max Activating Token Index: 440

Text #14

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8933. Min Act: -0.1700

Data Index: 8360181 (Open Web Text)

Max Activating Token Index: 609

Text #15

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8922. Min Act: -0.1700

Data Index: 137455 (Open Web Text)

Max Activating Token Index: 307

Text #16

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8927. Min Act: -0.1700

Data Index: 5088193 (Open Web Text)

Max Activating Token Index: 87

Text #17

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8739. Min Act: -0.1700

Data Index: 6170951 (Open Web Text)

Max Activating Token Index: 675

Text #18

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8857. Min Act: -0.1700

Data Index: 3987210 (Open Web Text)

Max Activating Token Index: 193

Text #19

Max Range: 1.9553. Min Range: -1.9553

Max Act: 1.8730. Min Act: -0.1700

Data Index: 5015121 (Open Web Text)

Max Activating Token Index: 437