Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 539 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.5914. Min Act: -0.1700

Data Index: 3852849 (Open Web Text)

Max Activating Token Index: 327

Text #1

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.5687. Min Act: -0.1700

Data Index: 7607218 (Open Web Text)

Max Activating Token Index: 308

Text #2

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.5376. Min Act: -0.1700

Data Index: 1072575 (Open Web Text)

Max Activating Token Index: 743

Text #3

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.4935. Min Act: -0.1700

Data Index: 31421 (Open Web Text)

Max Activating Token Index: 309

Text #4

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.5129. Min Act: -0.1700

Data Index: 6667175 (Open Web Text)

Max Activating Token Index: 547

Text #5

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.4999. Min Act: -0.1700

Data Index: 6824817 (Open Web Text)

Max Activating Token Index: 841

Text #6

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.4439. Min Act: -0.1700

Data Index: 8026652 (Open Web Text)

Max Activating Token Index: 517

Text #7

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.4368. Min Act: -0.1700

Data Index: 4080606 (Open Web Text)

Max Activating Token Index: 364

Text #8

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.4472. Min Act: -0.1700

Data Index: 4441052 (Open Web Text)

Max Activating Token Index: 776

Text #9

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.4198. Min Act: -0.1700

Data Index: 6107417 (Open Web Text)

Max Activating Token Index: 235

Text #10

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3955. Min Act: -0.1700

Data Index: 6299593 (Open Web Text)

Max Activating Token Index: 915

Text #11

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.4017. Min Act: -0.1700

Data Index: 485980 (Open Web Text)

Max Activating Token Index: 412

Text #12

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3539. Min Act: -0.1700

Data Index: 4882589 (Open Web Text)

Max Activating Token Index: 381

Text #13

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3863. Min Act: -0.1700

Data Index: 1658497 (Open Web Text)

Max Activating Token Index: 867

Text #14

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3299. Min Act: -0.1700

Data Index: 2067790 (Open Web Text)

Max Activating Token Index: 187

Text #15

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3294. Min Act: -0.1700

Data Index: 4080773 (Open Web Text)

Max Activating Token Index: 218

Text #16

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3629. Min Act: -0.1700

Data Index: 4354814 (Open Web Text)

Max Activating Token Index: 485

Text #17

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3576. Min Act: -0.1700

Data Index: 4958534 (Open Web Text)

Max Activating Token Index: 947

Text #18

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3105. Min Act: -0.1700

Data Index: 4372629 (Open Web Text)

Max Activating Token Index: 529

Text #19

Max Range: 4.5914. Min Range: -4.5914

Max Act: 4.3321. Min Act: -0.1700

Data Index: 2425769 (Open Web Text)

Max Activating Token Index: 563