If government spends more than it earns it has to plug the gap by borrowing money, so it issues bonds. These must be repaid, but the risk of default is very low for developed economies that can print their own money. The largest government bond market is the US ($16 trillion in Q3 2016) followed by Japan ($11 trillion) with developed European countries trailing far behind (UK $2.7 trillion, France $2.1 trillion, Italy $2.1 trillion, Germany $1.8 trillion, Spain $1.1 trillion).
For emerging market economies defaults are much more likely, so they pay a higher yield than developed market government bonds and this extra yield is to compensate investors for their higher default risk. Some emerging market countries issue debt in dollars which can lead to crises if the dollar strengthens as this increases both the level of their debt and their regular coupon payments.