C++ STL with jemalloc-Collection of common programming errors

How is it possible to use C++ STL containers with jemalloc (or any other malloc implementation)?

Is it as simple as include jemalloc/jemalloc.h? Or should I write an allocator for them?

Edit: The application I’m working on allocates and frees relatively small objects over its lifetime. I want the replace the default allocator, because benchmarks showed that the application doesn’t scale beyond 2 cores. Profiling showed that it was waiting for memory allocation, that’s what caused the scaling issues. As I understand, jemalloc will help with that.

I’d like to see a solution, that’s platform-neutral as the application has to work on both Linux and Windows. (Linking against a different implementation is easy under Linux, but it’s very hard on Windows as far as I know.)