How to prevent sendmail from losing mail when remote host is unreachable

I have a small domain with multiple hosts. Sendmail is set up to forward all locally generated mail (mostly mail to root@localhost generated by cron jobs) to a central "mail hub", using

define(`MAIL_HUB', `somehost.mydomain.org')

in sendmail.mc. Recently I had a problem with the mail server on the remote end, so that It would not accept mail from this host. This caused sendmail to drop all mail. From /var/log/maillog:

sendmail[3133]: n4461S5s003133: n4461S5t003133: return to sender: Service unavailable
sendmail[3133]: n4461S5t003133: to=root, delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=34772, relay=XXXXX.org.  [91.184.38.153], dsn=5.0.0, stat=Service unavailable
sendmail[3133]: n4461S5s003133: Losing ./qfn4461S5s003133: savemail panic

All mails generated during this time were dropped and lost forever.

How can I prevent this? Ideally I'd like sendmail to try again laiter or eventually to save the mail to a local file as a last resort.

From serverfault 8jean

• Your server almost did the "correct" thing here.

  The far end returned a "5xx" class error message, indicating a permanent failure to deliver the message, hence your end aborted delivery completely.

  For your server to continue to queue the message it would have had to have received a "4xx" class message, indicating a temporary failure.

  To prevent the messages from being lost forever you do however need to identify what caused the "savemail panic" error - they should have just been bounced back to the senders.

  8jean : The last paragraph summarizes my problem: both senders and recievers are local accounts, for which somehost.mydomain.org is the final destination (says "MAIL_HUB"). I'm starting to assume the "MAIL_HUB" option is only suitable for setups where either the mail hub for a domain is guaranteed to be up, or it's ok to lose local mail on the other hosts.
  From Alnitak

• Generally, sendmail should save any failed send attempt locally in spool files and retry for up to five days, so probably there is some configuration error. According to these references:

  • savemail panic in sendmail
  • sendmail: savemail panic

  you need to make sure that you don't have (from above references):

  • Missing postmaster alias in /etc/aliases
  • Hard disk is full
  • The mail spool for the postmaster has the wrong ownership
  • The mbox file for the postmaster is over 2GB and procmail can’t deliver the e-mail

  Additionally, ensure that you're not running into an selinux issue. It may be that selinux is preventing sendmail from saving the file. EMail should never be totally lost -- at worst it should be bounced to the original sender and to postmaster. If EMail is totally lost, it's either a configuration error or a bug in sendmail.

  What this error means, in context, is that sendmail tried to deliver the message in spool file ./qfn4461S5s003133 but the remote mail server returned the error response "Service unavailable." Thus, the local sendmail attempted to bounce the EMail. Something went wrong while trying to do so, so the EMail was abandoned.

  This is the EMail equivalent of a CPU's double bus fault. That is, a fatal error (not being able to save or bounce the EMail message) occurring in response to a fatal error (the remote server being unavailable for long enough that sendmail gave up retrying).

  8jean : Thanks, this pointed me to the right direction. The problem is that "MAIL_HUB" sets the destination for all local mail (including mail tor root or postmaster). So sendmail does indeed seem to do the right thing. Perhaps I need to rethink my configuration.
  From Eddie