Random thoughts on optimizing code for scientific computations

I have no time to write an essay, so here are some fragmentary thoughts.

  1. Scientists who deal with computationally demanding research subjects can be obsessed with code performance, and sometimes rightly so. However, this obsession, if not tamed by sound principles and pragmatic wisdom, may lead us down the evil path of Premature Optimization. We should always be reminded of its danger. The aphorism by Donald Knuth may be recited three times a day:

    Premature optimization is the root of all evil.

  2. Make it work first, and then make it fast.

  3. Follow the Rules Of Optimization.

  4. Level of trust: profiler > micro-benchmark tests > design for performance.

  5. Refactoring is a never-ending quest. You can make the work of the future “you” easier by writing clean and maintainable code at this moment.

  6. There is not a language that rules them all. Even in the small niche of scientific computing. Never will. Learn multiple languages and pick the best horse for the course.

  7. Do not idolize Fortran (or C or C++). If you don't have tight inner loops that consume 80% of your computation time, then you probably don't need to use them (unless you enjoy so). Vita brevis; learn and use a high-level language.

Afterthought: I think some of the rules of optimization also apply to writing a manuscript. For example, do not commit premature editing before having the first draft.

References

  1. “CategoryOptimization”, http://wiki.c2.com/?CategoryOptimization. This contains a bunch of insightful posts on how to deal with optimization.
  2. Fowler, M. (2002). Yet Another Optimization Article. IEEE Software, 19(3), 20–21. doi:10.1109/MS.2002.1003448. Retrieved at https://www.martinfowler.com/ieeeSoftware/yetOptimization.pdf on 2018-06-13.