{"id":7506,"date":"2015-07-01T00:45:01","date_gmt":"2015-07-01T00:45:01","guid":{"rendered":"https:\/\/unknownerror.org\/index.php\/2015\/07\/01\/recovering-cleanly-from-resquetermexception-or-sigterm-on-heroku\/"},"modified":"2015-07-01T00:45:01","modified_gmt":"2015-07-01T00:45:01","slug":"recovering-cleanly-from-resquetermexception-or-sigterm-on-heroku","status":"publish","type":"post","link":"https:\/\/unknownerror.org\/index.php\/2015\/07\/01\/recovering-cleanly-from-resquetermexception-or-sigterm-on-heroku\/","title":{"rendered":"Recovering cleanly from Resque::TermException or SIGTERM on Heroku"},"content":{"rendered":"<p>When we restart or deploy we get a number of Resque jobs in the failed queue with either <code>Resque::TermException (SIGTERM)<\/code> or <code>Resque::DirtyExit<\/code>.<\/p>\n<p>We&#8217;re using the new <code>TERM_CHILD=1 RESQUE_TERM_TIMEOUT=10<\/code> in our Procfile so our worker line looks like:<\/p>\n<pre><code>worker:  TERM_CHILD=1 RESQUE_TERM_TIMEOUT=10 bundle exec rake environment resque:work QUEUE=critical,high,low\n<\/code><\/pre>\n<p>We&#8217;re also using <code>resque-retry<\/code> which I thought might auto-retry on these two exceptions? But it seems to not be.<\/p>\n<p>So I guess two questions:<\/p>\n<ol>\n<li>We could manually rescue from <code>Resque::TermException<\/code> in each job, and use this to reschedule the job. But is there a clean way to do this for all jobs? Even a monkey patch.<\/li>\n<li>Shouldn&#8217;t resque-retry auto retry these? Can you think of any reason why it wouldn&#8217;t be?<\/li>\n<\/ol>\n<p>Thanks!<\/p>\n<p><strong>Edit:<\/strong> Getting all jobs to complete in less than 10 seconds seems unreasonable at scale. It seems like there needs to be a way to automatically re-queue these jobs when the Resque::DirtyExit exception is run.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>When we restart or deploy we get a number of Resque jobs in the failed queue with either Resque::TermException (SIGTERM) or Resque::DirtyExit. We&#8217;re using the new TERM_CHILD=1 RESQUE_TERM_TIMEOUT=10 in our Procfile so our worker line looks like: worker: TERM_CHILD=1 RESQUE_TERM_TIMEOUT=10 bundle exec rake environment resque:work QUEUE=critical,high,low We&#8217;re also using resque-retry which I thought might auto-retry [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-7506","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7506","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=7506"}],"version-history":[{"count":0,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/7506\/revisions"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=7506"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=7506"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=7506"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}