{"id":3966,"date":"2014-03-30T06:44:49","date_gmt":"2014-03-30T06:44:49","guid":{"rendered":"https:\/\/unknownerror.org\/index.php\/2014\/03\/30\/problem-about-memory-mapped-files-collection-of-common-programming-errors\/"},"modified":"2014-03-30T06:44:49","modified_gmt":"2014-03-30T06:44:49","slug":"problem-about-memory-mapped-files-collection-of-common-programming-errors","status":"publish","type":"post","link":"https:\/\/unknownerror.org\/index.php\/2014\/03\/30\/problem-about-memory-mapped-files-collection-of-common-programming-errors\/","title":{"rendered":"problem about memory-mapped-files-Collection of common programming errors"},"content":{"rendered":"<ul>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/2a04086d7656ac97e09c20c4fe4d8266?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nsharptooth<br \/>\nwindows winapi memory-mapped-files<br \/>\nWhat are the drawbacks (if any) of using memory mapped file to read (regular sized files) over doing the same using CreateFile ReadFile combination?<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/36b6fe38a09145199ad3a8b0b706b136?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nAndrew Harry<br \/>\nc#-4.0 memory-mapped-files rdms<br \/>\nI&#8217;m interested in people&#8217;s thoughts comparing storing data in a traditional SQL based Database or utilising a Memory-Mapped File such as the one in the new .Net 4.0 runtime. The data in question would be arrays of simple structures.Obvious pros and cons:SQL Database ProsAdhoc query support SQL Management Tools Schema changes (adding more columns and setting default values)Memory-Mapped ProsLighter overhead? (this is an assumption on my part) Shareable between process threads Any others?Is it wo<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/i.stack.imgur.com\/YiNTB.jpg?s=32&amp;g=1\" \/><br \/>\nOopsUser<br \/>\nc++ file boost memory-mapped-files<br \/>\nImportant edit : The problem is not what i stated, after manually profiling i understood that when i replace the line : &#8220;file &gt;&gt; x &gt;&gt; y &gt;&gt; z;&#8221; with the line &#8220;file.readline(buffer, size);&#8221;it takes only 0.4 seconds, so the question is entirely different, how to parse the floats from the line, file&gt;&gt;x&gt;&gt;y&gt;&gt;z;(i don&#8217;t know if i should delete the question or not, because the original question is not relevant)=== OLD === After vast research on the internet and stack overflow, i understood that the best<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/31dd8dd7471ddfdc497f4081b7f01c2b?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nRob Kielty<br \/>\njava centos memory-mapped-files tlb memory-barriers<br \/>\nMemory barriers guarantee that the data cache will be consistent. However, does it guarantee that the TLB will be consistent?I am seeing a problem where the JVM (java 7 update 1) sometimes crashes with memory errors (SIGBUS, SIGSEG) when passing a MappedByteBuffer between threads.e.g.final AtomicReference&lt;MappedByteBuffer&gt; mbbQueue = new AtomicReference&lt;&gt;();\/\/ in a background thread. MappedByteBuffer map = raf.map(MapMode.READ_WRITE, offset, allocationSize); Thread.yield(); while (!i<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/4d4c5200e64564047976e390fa9ed1cc?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nCratylus<br \/>\njava tcp io nio memory-mapped-files<br \/>\nI am interested in implemented the following simple flow: Client sends to a server process a simple message which the server stores. Since the message does not have any hierarchical structure IMO the best approach is to save it in a file instead of an rdb. But I want to figure out how to optimize this since as I see it there are 2 choices: Server sends a 200 OK to the client and then stores the message so the client does not notice any delay Server saves the message and then sends the 200OK but<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/d590f7c181f85e9a109339c22110e4d2?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nAan<br \/>\nc++ winapi memory-leaks memory-mapped-files memory-mapping<br \/>\nI am trying to use memory-mapped files as:hFile = ::CreateFile(State.Path, GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE,0, OPEN_EXISTING,FILE_FLAG_SEQUENTIAL_SCAN, 0);\/\/open the fileif(hFile !=INVALID_HANDLE_VALUE){ hMap= ::CreateFileMapping(hFile, 0, PAGE_READONLY | SEC_COMMIT, 0, 0, 0);\/\/create Mem mapping for the file in virtual memory if( hMap!=NULL){ base = ::MapViewOfFile(hMap, FILE_MAP_READ, 0, 0, 0);\/\/load the mapped file into the RAM \/\/start to compare some bytes (values) from mspai<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/a4b291b8d429dc2d50693fe212127590?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nVladimir Gritsenko<br \/>\nios nsdata memory-mapped-files<br \/>\nI have a file which I memory-map with NSData. I then delete this file via an NSFileManager without any error, and proceed to check that the file is indeed not there (as far as NSFileManager and ls are concerned). However, I can still read data from the bytes pointer I got from NSData previously!In the simulator, I sometimes get seemingly unrelated crashes. On the device, everything seems to work fine. I&#8217;m very curious to know what&#8217;s going on, and what should I expect (what I expected in the past<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/2385676760fc16f76a43f3319faaa843?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nJohannes Schaub &#8211; litb<br \/>\nc++ c qt mmap memory-mapped-files<br \/>\nI am using Qt to map a file to a piece of memory pagesQFile::map (qint64 offset, qint64 size, MemoryMapFlags flags = NoOptions)Essentially, this should be a mmap system function call. I wonder how I can guarantee that I can access the returned memory, even if the file on disk is truncated. I seem to need this because I read from a disk file and want to gracefully handle errorsif (offset &gt; m_file.size()) \/\/ throw an error&#8230; if (m_mappedFile != NULL) return m_mappedFile + offset;Obviously, thi<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/3283e7cedc26613d7f38c80fa27eb144?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nMario<br \/>\nc# mutex memory-mapped-files<br \/>\nI&#8217;m constantly reading from a memory mapped file another process is writing to and use a mutex to synchronize this operation. In my few tests so far this works just fine, but&#8230; what if my application crashes right after acquiring the mutex and before releasing it? Is there any way to guarantee a release of the mutex, even in case of such a crash?Also how would I handle a crash of the other process, which might not have released the mutex yet? Do I need to handle AbandonedMutexException each tim<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/8d7ecc33f727fe807606ebbe3dd8e2f0?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nHeather<br \/>\nc# serialization memory-mapped-files binaryformatter<br \/>\nI&#8217;m trying to serialize my MMF to file and here is the code:class MMF {private const string filename = @&#8221;c:\\NFS&#8221;;private long offset = 0;private long length = 194;private byte[] buffer;public MMF() {using (var mmf =MemoryMappedFile.CreateFromFile(filename, FileMode.OpenOrCreate, null, offset + length, MemoryMappedFileAccess.ReadWriteExecute)) {using (var accessor = mmf.CreateViewAccessor(offset, length, MemoryMappedFileAccess.ReadWriteExecute)) {buffer = new byte[194];\/*FS fs = new FS();fs.Files<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/e560af70c1e280d80a5c7d3540ed6bed?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nKieren Johnstone<br \/>\nc# performance debugging .net-4.0 memory-mapped-files<br \/>\nI have a client\/server app. The server component runs, uses WCF in a &#8216;remoting&#8217; fashion (binary formatter, session objects).If I start the server component and launch the client, the first task the server does completes in &lt;0.5sec.If I start the server component with VS debugger attached, and then launch the client, the task takes upwards of 20sec to complete.There are no code changes &#8211; no conditional compilation changes. The same occurs whether I have the server component compiled and runn<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/7c52a3944cdd2eac9725e54fc548f368?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nuser997112<br \/>\nc++ boost memory-mapped-files<br \/>\nI am using the below code to open files which are approximately 400 to 800MB in size:#include &lt;boost\\interprocess\\file_mapping.hpp&gt; #include &lt;boost\\interprocess\\mapped_region.hpp&gt; #include &lt;iostream&gt; #include &lt;vector&gt; #include &lt;string&gt;using namespace boost::interprocess; using namespace std;int main(){file_mapping fm(&#8220;C:\\\\test\\\\1.txt&#8221;,read_only);mapped_region region(fm,read_only);const char* const data = static_cast&lt;const char*&gt;(region.get_address());const siz<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/206690a26526f07467ecfd6662f8b152?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nKarussell<br \/>\njava nio mmap memory-mapped-files<br \/>\nI&#8217;ve a small and simple storage system which is accessible through memory mapped files. As I need to address more than 2GB space I need a list of MappedByteBuffer with a fixed size like 2GB (I&#8217;m using less for different reasons). Then all is relativly simple: a buffer maps to a certain space say to 1GB and when I need more I map a new MappedByteBuffer (the file automatically increases), then when I need more a third buffer is mapped etc. This just worked.But then I&#8217;ve read in the Java NIO book t<\/li>\n<li><img decoding=\"async\" src=\"http:\/\/www.gravatar.com\/avatar\/f64914b68ff9076e46156c07c96e890f?s=32&amp;d=identicon&amp;r=PG\" \/><br \/>\nR. Martinho Fernandes<br \/>\nc++ persistence memory-mapped-files pod<br \/>\nPointers cannot be persisted directly to file, because they point to absolute addresses. To address this issue I wrote a relative_ptr template that holds an offset instead of an absolute address.Based on the fact that only trivially copyable types can be safely copied bit-by-bit, I made the assumption that this type needed to be trivially copyable to be safely persisted in a memory-mapped file and retrieved later on.This restriction turned out to be a bit problematic, because the compiler genera<\/li>\n<\/ul>\n<p>Web site is in building<\/p>\n","protected":false},"excerpt":{"rendered":"<p>sharptooth windows winapi memory-mapped-files What are the drawbacks (if any) of using memory mapped file to read (regular sized files) over doing the same using CreateFile ReadFile combination? Andrew Harry c#-4.0 memory-mapped-files rdms I&#8217;m interested in people&#8217;s thoughts comparing storing data in a traditional SQL based Database or utilising a Memory-Mapped File such as the [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3966","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/3966","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/comments?post=3966"}],"version-history":[{"count":0,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/posts\/3966\/revisions"}],"wp:attachment":[{"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/media?parent=3966"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/categories?post=3966"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unknownerror.org\/index.php\/wp-json\/wp\/v2\/tags?post=3966"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}