When then master points to the horizon the fool looks at the finger
When we solve a problem we analyze the lay of the land. We try our best to understand the requirements: what people need from the system. All too often we fail to write good specifications for a system that meets those requirements. But we do write a lot of code.
Throughout your career you might be accused of over-engineering. You picked a library that people find difficult to understand. You used language features that are considered too advanced for new programmers. You planned for features that you aren’t going to use. All of these and more are the different justifications people give when they judge that your solution is over-engineered.
Don’t be caught by short-sighted fools! If you have taken care to understand the problem and bring many years of experience to a team then they should take more care before accusing you of over-engineering. Often times fore-sight and experience are mistaken for over-engineering. As are using advanced language features or uncommon programming practices and patterns!
Wikipedia defines over-engineering as:
the act of designing a product or providing a solution to a problem in an elaborate or complicated manner, where a simpler solution can be demonstrated to exist with the same efficiency and effectiveness as that of the original design.
They give famous examples like the Panther tank from Germany during the Second World War. These tanks used expensive materials and were difficult to produce. This limited their production and made them hard to repair in the field.
Bikes are another interesting example. You can tell a bike is over-engineered when it has features that make it difficult to operate for its intended use. When the bike itself breaks down and the user cannot do anything to repair it without taking it to a specialist the bike might be over-engineered.
It depends on what the requirements are. If the bike is intended for quick commutes across town then it would be over-engineered: the user may encounter situations where they need to repair the bike and they do not have a specialist on hand to repair it for them. However for a long distance race you may need a more sophisticated derailleur setup. And you’re likely to have the specialist nearby when it breaks down.
Building the Wrong Thing
Fortunately for us software does not have similar mechanical limitations as bikes and tanks.
Unfortunately for us this means that it’s not clear when a piece of software is over-engineered.
A developer of a platforming game might ask themselves, Did I program this jump code right? If we look at the specification for the program we might find some answers. It must be fun and feel good. Do you have fun when you make the character jump? If the answer is yes then the program is right. You will gain nothing by implementing the game using “proper” physics in the code or adhering to a programming paradigm that is considered a, “best practice.”
The code can be as messy or clean as it is. All that matters is that it’s fun.
Code is malleable. If you write some library of code that solves all kinds of problems in a super generic way then all you have done is perhaps wasted a bit of time and money. In the end if the program meets the specification then it’s fine. You can always refactor in order to change the code in other dimensions your team cares about: readability, maintainability, etc.
You have more to lose from building the wrong thing than you do from over engineering.
Where is the Line?
Fortunately, unlike a Panther tank or a Mercedes-Benz, the cost of over-engineering in software development is not as high and immediate. However it still comes with a cost.
The more features we add the more complex our designs. This gives us a larger surface area for errors. And code gets complex fast. It can sometimes seem like we are fighting a constant battle with complexity.
The key I find is to use what is sufficient to solve the problem. Part of that is knowing when to write specifications and how precise those specifications need to be. The more likely things are to go wrong and the higher the cost of failures are then the more precise we ought to be. This includes using language features that allow us to express parts of our specifications with static typing and analysis.
We must be careful not to conflate complexity with difficulty. Some problems require solutions that may be difficult to understand for someone who is not familiar with the problem, the language, the tools, etc. The use of certain language features exist in order to help us write more correct programs. Correct, with respect to our specifications.
Finally, remember that much software is malleable. The cost of over-engineering in software is some times over stated. You can cut down the number of features, you can simplify interfaces, and you can eliminate unnecessary code. It’s great if we can avoid introducing extra code to begin with but it shouldn’t prevent us from making progress.
Unless of course you’re writing firmware that will be burned onto a ROM chip. Or you’re writing critical control software. It may pay off to be right the first time in these cases. And more care must be taken before writing code.
Think about your requirements, try to define your specifications, and design your solutions to those specifications. Don’t worry so much about over-engineering. Try to avoid it if you can.
And remember: even chasing maximal simplicity is itself a form of over-engineering. It’s all about trade offs.