{"id":6632,"date":"2014-04-20T13:58:09","date_gmt":"2014-04-20T13:58:09","guid":{"rendered":"https:\/\/unknownerror.org\/index.php\/2014\/04\/20\/wget-and-connection-errors-timeouts-collection-of-common-programming-errors\/"},"modified":"2014-04-20T13:58:09","modified_gmt":"2014-04-20T13:58:09","slug":"wget-and-connection-errors-timeouts-collection-of-common-programming-errors","status":"publish","type":"post","link":"https:\/\/unknownerror.org\/index.php\/2014\/04\/20\/wget-and-connection-errors-timeouts-collection-of-common-programming-errors\/","title":{"rendered":"wget and connection errors \/ timeouts-Collection of common programming errors"},"content":{"rendered":"<p>I was using wget the in the last week to recursively download a whole website of html pages. I used it this way:<\/p>\n<pre><code>wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains XXXX.com --no-parent http:\/\/www.XXXX.com\n<\/code><\/pre>\n<p>the issue is, since the downloading took couple of days, sometimes there were connection timeouts , network disconnections etc, and while it happened, seems like wget skipped the htmls it couldnt fetch, which is not good in this case.<\/p>\n<p>I wonder if there is a flag (been looking in the manpage to no avail&#8230;) to tell wget to keep on retrying failed fetches for infinity? (even if my computer disconnects for the web for 10 hours, I want it to keep on trying to get the page until it succeeded (obviously when the computer will be online again).<\/p>\n<p>thanks,<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I was using wget the in the last week to recursively download a whole website of html pages. I used it this way: wget &#8211;recursive &#8211;no-clobber &#8211;page-requisites &#8211;html-extension &#8211;convert-links &#8211;domains XXXX.com &#8211;no-parent http:\/\/www.XXXX.com the issue is, since the downloading took couple of days, sometimes there were connection timeouts , network disconnections etc, and while it [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-6632","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/6632","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=6632"}],"version-history":[{"count":0,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/6632\/revisions"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=6632"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=6632"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=6632"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}