How to prevent node.js from crashing when opening too many files at once?-Collection of common programming errors
Not sure if this is the best way to solve this issue, but what I’ve effectively done is wrapped all the asynchronous fs functions with a wrapper that checks whether or not the we have more than the allowed amount of files open (arbitrarily set to a reasonable 100). If there is more than that many files open at the time of the call, the call will idle and check again after a delay to see if there is now an opening. The code for this logic can be found here: https://github.com/balupton/bal-util/blob/master/src/lib/paths.coffee#L7-45
In terms of implementing it, you just need to change require('fs').readFile
or whatever call you are using to require('bal-util').readFile
and install the bal-util dependency (npm install bal-util
) which is simple enough.
Hope this helps. If there is a more robust way of doing this, I’d love to know.