Every modern democratic state imprisons thousands of offenders every year, depriving them of their liberty, causing them a great deal of psychological and sometimes physical harm. Relationships are destroyed, jobs are lost, the risk of the offender being harmed by other offenders is increased and all at great expense to the state. How can this brutal and costly enterprise be justified? Traditionally, philosophers answering this question have argued either that the punishment of wrongdoers is a good in itself (retributivism), or that it is a regrettable means to a valuable end, such as the deterrence of future wrongdoing, and thus justifiable on consequentialist grounds. This book offers a critical examination of those theories and advances a new argument for punishment's justification, calling it the 'duty view'. On this view, the permission to punish offenders is grounded in the duties that they incur in virtue of their wrongdoing. The most important duties that ground the justification of punishment are the duty to recognize that the offender has done wrong and the duty to protect others against wrongdoing. In the light of these duties the state has a permission to punish offenders to ensure that they recognize that what they have done is wrong, but also to protect others from crime. In contrast to other justifications of punishment grounded in deterrence, the duty view is developed in the light of a non-consequentialist moral theory: a theory which endorses constraints on the pursuit of the good. It is shown that it is normally wrong to harm a person as a means to pursue a greater good. However, there are exceptions to this principle in cases where the person harmed has an enforceable duty to pursue the good. The implications of this idea are explored both in the context of self-defence, and then in the context of punishment. Through the systematic exploration of the relationship between self-defence and punishment, the book makes significant progress in defending a plausible set of non-consequentialist moral principles that justify the punishment of wrongdoers, and marks a significant contribution to the philosophical literature on punishment.