As @QFMathew mentioned you will need to first unlink the invoice, then the option will appear to delete it.