Vagrant crashes depending on physical network-Collection of common programming errors

I have a virtual machine I manage using Vagrant. When provisioning, I get some updates for the system (Ubuntu) using apt-get, install Node.js using nvm, and then run npm install for various modules. Now strange things happen:

If I try to create and provision the VM in physical network A (at one place), provisioning fails. Either creation already fails and Vagrant tells me that the machine went to an invalid state, or the VM crashes when npm starts to do its work.

If I try to do the exactly same thing on physical network B (at another place), everything is fine.

As the only difference between the two places (I can think of) is the physical network, I wonder how this can happen. I have tried it with two different MacBooks, one running Mavericks, the other running Mountain Lion. The effect happens on both and also when using different Vagrant / VirtualBox versions, so apparently it’s actually a problem of the environment.

Any idea what might cause issues like that?