Optimize performance

Before running simulations on a large scale, it might be worthwhile to first optimize the performance. This section gives some advice on how to do so. See also the Python wiki pages on this topic: https://wiki.python.org/moin/PythonSpeed.

CPU

  • The first step in performance optimization is often to profile your code, for example using cProfile, which is included in the Python standard library.

  • Calculate as many values as possible in the properties() and properties_tmp() methods.

  • MISCore uses the NumPy SFC64 pseudorandom number generator, which runs significantly faster using the 64-bit Python interpreter, compared to the 32-bit interpreter. Thus, make sure to use the 64-bit version (if you have a 64-bit system).

  • Some parts of MISCore are optimized using Cython. If Warning: Optimized Individual could not be loaded is printed when running MISCore, this optimized version could not be properly loaded.

  • The utils built into MISCore are optimized using Cython. Use them in your processes where possible. You can find an overview of the Utils at Modify a process.

  • Log only the events and durations you actually need. Pass a list of event and durations tags to the run() method of Model to do so.

  • Log only events and durations in relevant intervals. Pass lists of age and year intervals to the run() method of Model to do so.

RAM

  • The block_size argument of run() determines for how many individuals at a time the temporary properties are calculated. RAM usage can be minimized by setting this argument to a small number. The default block_size is 2000.

  • The advice given regarding logging for CPU performance also holds for optimizing RAM performance.