{"id":5054,"date":"2014-03-30T18:24:53","date_gmt":"2014-03-30T18:24:53","guid":{"rendered":"https:\/\/unknownerror.org\/index.php\/2014\/03\/30\/64-bit-large-mallocs-collection-of-common-programming-errors\/"},"modified":"2014-03-30T18:24:53","modified_gmt":"2014-03-30T18:24:53","slug":"64-bit-large-mallocs-collection-of-common-programming-errors","status":"publish","type":"post","link":"https:\/\/unknownerror.org\/index.php\/2014\/03\/30\/64-bit-large-mallocs-collection-of-common-programming-errors\/","title":{"rendered":"64 bit large mallocs-Collection of common programming errors"},"content":{"rendered":"<p>Here&#8217;s an official source that states the maximum request size of the heap is defined by your linked CRT library (aside from your previous code having integer overflows going to 0 which is why you didn&#8217;t get NULL back) (_HEAP_MAXREQ).<\/p>\n<p>http:\/\/msdn.microsoft.com\/en-us\/library\/6ewkz86d.aspx<\/p>\n<p>Check out my answer here for large windows allocations, I include a reference to a MS paper on Vista\/2008 memory model advancements.<\/p>\n<p>In short, the stock CRT does not support, even for a native 64 bit process any heap size larger than 4gb. You <em>have<\/em> to use VirtualAlloc* or CreateFileMapping or some other analogues.<\/p>\n<p>Oh I also noticed you are claiming that your larger allocations are actually succeeding, this is actually incorrect, you are mis-interpreting the malloc(0x200000000); (that&#8217;s 8gb in hex), what is happening is you are requesting a 0 byte allocation due to a cast or some other effect of your test harness, you are most definitely not observing any thing larger than a 0xfffff000 bytes heap being committed, it is simply you are seeing integer overflows down casting.<\/p>\n<p><strong>WORD TO THE WYSE<\/strong> or * TIPS TO SAVE YOUR HEAP SANITY*<\/p>\n<h2>THE <strong><em>ONLY<\/em><\/strong> WAY TO ALLOCATE MEMORY WITH MALLOC (OR ANY OTHER DYNAMIC REQUEST)<\/h2>\n<pre><code>void *foo = malloc(SIZE);\n<\/code><\/pre>\n<h2>THE VALUE OF A DYNAMIC MEMORY REQUEST MUST NEVER (I CAN NOT STRESS THAT ENOUGH) BE CALCULATED WITHIN THE &#8220;()&#8221; PAREN&#8217;S OF THE REQUEST<\/h2>\n<pre><code>mytype *foo = (mytype *) malloc(sizeof(mytype) * 2);\n<\/code><\/pre>\n<p>The danger is that an <strong>integer<\/strong> overflow will occur.<\/p>\n<p>It is always a coding <strong>ERROR<\/strong> to perform arithmetic at the time of the call, you <strong>MUST ALWAYS<\/strong> calculate the <strong>TOTAL SUM<\/strong> of data to be requested before the statement which evaluates the request.<\/p>\n<p>Why is it so bad? We know this is a mistake, because the point at which a <em>request<\/em> is made for dynamic resources, there <strong>must<\/strong> be a point in the future where we will <em>use<\/em> this resource.<\/p>\n<p>To use what we have requested we <strong>must<\/strong> know how large it is ? (e.g. the array count, the type size, etc..).<\/p>\n<p>This would mean, if we ever see any arithmetic at all, inside the () of a resource request, it is an error as we <strong>MUST<\/strong> duplicate that code <em>again<\/em> in order to use that data appropriately.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Here&#8217;s an official source that states the maximum request size of the heap is defined by your linked CRT library (aside from your previous code having integer overflows going to 0 which is why you didn&#8217;t get NULL back) (_HEAP_MAXREQ). http:\/\/msdn.microsoft.com\/en-us\/library\/6ewkz86d.aspx Check out my answer here for large windows allocations, I include a reference to [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-5054","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/5054","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=5054"}],"version-history":[{"count":0,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/5054\/revisions"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=5054"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=5054"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=5054"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}