The Trade – 4 ways to (Eventually) do the right thing

To every decision there is a dark side, this an unshakable fact of life.

  • The bus is cheaper than a car but takes more time.
  • Going for a run is good for your health, but you my get injured
  • Having a smart phone enables you to work from everywhere, but you may end up working from everywhere

Tech in that perspective is no different, you often hear about “premature optimization”1 and “design bugs”2. These are merely bad choices – made without understanding the trade offs  between the available options. It gets worse, sometimes there is no perfect choice out there to be found, because your options are limited by what you can actually do.

  • A private cloud is more secure but way more expensive.
  • Developing everything in-house gives you more control but takes a lot more time (and workers, and money)
  • Haskell is best for this project but the client will never be able to self-maintain it

Of course, it’s the basically same for every gadget and software acquisition, in every stack choice and in every design decision you’ll ever make, but that is overwhelming and you can’t get anything done if you’ll just sit there decision fatigued and  annoyed.

This is why the Agile management systems were invented, to make you feel that it’s OK to make mistakes (which it is3) and that everything is reversible (which it isn’t). However the truth is, some decisions you’ll take will prevent you from doing things you may want to do later on. Nowhere it is more true than in the field of performance optimization.

  • Apache Storm will manage your processes for you,  only if these processes are independent
  • Using the python “map” function will better utilize your processors, but the code will have to be heavily refactored
  • Some assumptions you can take will enable you to use some faster algorithms, but what if tomorrow they will be wrong?
  • More processors per node or more nodes per cluster?

All these should be taken into account, some you could decide for yourself and some will have to be discussed (choosing wrong here will land you back at footnote 3), yet the worst kind is the trade-off no-one chose. I bet you have some of those lying around your system and you never realized, And one day, like any other bug, they will jump at you from the dark screaming and with bloodshot eyes demanding your undivided attention. On the other hand, they may lay there forever, enjoying the fact you can’t test for them destroying your performance day after day, request after request.

So now that I have you totally terrified I’ll share with you

be thorough and proactive

Look for other options, look for what have been done before in your organisation, look for an expert, very rarely you’ll find yourself on uncharted lands.


Talk to your coworkers, talk to the client, talk to your friends, talk to your mom. Explain them the problem in as simplest terms as you can and explain your solution, and ask them what they think4. The more you process it the more likely it is that you’ll think of something that haven’t occurred to you earlier.

have confidence in yourself

Once you made you made up your mind go on with confidence, you can sleep well knowing you’ve done your research and  your  “hallway testing”, and finished all your vegetables.

But be humble And flexible

You have the best solution? So says you. Be open to the notion that you may be wrong, you probably did. Be prepared fix it. Whatever it takes. Don’t hang up on your old code, as much as you love it, if it holds you back you got to delete that sucker from existence. and if someone else found a better solution, smile, you may be proven wrong, but at least now you know for sure there was a better solution.


  1. The act of optimizing buggy code – rendering it virtually undebugable
  2. The origin of these is usually a failure to account for all types of changes the system will need to undergo in the future.
  3. Also, it is OK to fail, to be yelled at, and to lose your job. Believe me, these are all terrible but invaluable experiences, and i have yet to see someone to come out the other side of one of them knowing less about his occupation, and about himself.
  4. And listen!

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

The Trade – 4 ways to (Eventually) do the right thing

Leave a Reply

Your email address will not be published. Required fields are marked *