A stigma is something to mark infamy or disgrace.
Many consider having superpowers to be a stigma. This is particularly true of mutants. There are those whose mutations have made them physically inhuman, like the Morlocks, and therefore cannot integrate in society. There are also those who hate and fear mutants who feel any power is a stigma. Galactus allowed Silver Surfer to keep the Power Cosmic so that wherever he went he would face the stigma of having served him.