Skip to Content

My Software Engineering Philosophy

And why it works

My basic approach to software engineering is waste management.

I've worked in development teams that were extremely high-pressure, with constant demand for new things and everything being rushed.  There was lots of overtime, lots of stress, many arguments, and very little actually got produced.

Just before I left this role, I did an analysis of our git history to compute the lines of code that got written vs what actually went into production.  My findings were scary.  I calculated that some 80% of the code written was never even deployed.  Based on some rough estimates of the team I was managing and their salaries this came out to some $300,000 in total.

The basic cause of the issues was the fact that everything was so rushed constantly, that a lot of mistakes were made - both at the engineering level but also at the product development level.  Things weren't thought through and while we were always incredibly busy, we weren't very productive.

I could probably write a small book on that period of my life, but the main lesson I learned there is that you get more done on a software team, not by pressuring them to "work harder" or "get things done", but on removing sources of waste.  Here's why.

Software is hard

It's complex, takes a lot of mental energy and is generally quite difficult to get right.

Imaging standing over a brain surgeon and pressuring them every 5 minutes to get the surgery done faster.  You are unlikely to actually get it done any faster than just leaving the guy alone.  If he nicks an artery, or stabs the scalpel into the wrong part of the brain, then you've just gone and created 10 times the work to try and patch this guy up than you already had before.

Also most engineers are extremely passionate.  Honestly you wouldn't last very long writing software if you don't really enjoy it.  So it's very rare that the actual issue is a lack of motivation.  And if it is, pushing people harder is rarely the solution to that.

If you don't trust the engineers to be motivated and working hard on the problems, then you've hired the wrong people.  The solution is not to micro-manage them and constantly pressure them for status updates.  The solution is to hire people you trust and let them get on with it.

Then create motivation by listening to their ideas, having stimulating discussions, providing training and opportunities, and spending time communicating the purpose of their work and how it helps the business or the customer.

People have an innate desire to help each other, so tap into that rather than bring out your big stick and start hitting people with it.

People don't know what they want

Ok so you get your management culture right, but you can still end up with lots of waste.

Often non-technical people (product manager, stakeholders, customers etc) don't really understand what they want.  And they also lack the language to describe it.

Engineers typically are not very knowledgeable about the business domain they are operating in.  If you are writing accounting software for example, it's highly unlikely you are also a fully trained, chartered accountant.

So one thing I always do is encourage my engineers to spend time trying to understand the problems they are working on, because the solution they build will be limited by their understanding of the problem.

This is also where the idea of early prototyping comes in.  You can be told that someone wants a new feature, and only after you build it they start asking for a bunch of stuff they never mentioned before.

Showing them a wire-frame early on, or just building out the front-end first and getting them to review it before you spend time on expensive back-end code, can flush out these requirements that people didn't even realise they had.

Also sometimes you need to put to one side what you've been asked for, and try to understand what they are trying to achieve.

Techniques like the 5-whys help you drill down into what the actual problem is and often you can think of much better solutions than what you've been asked to build.

Imagine as an engineer you are asked to build a catapult that can fling cars over a 2km distance, going straight into designing and building this without finding out why you've been asked for it would be a mistake.  Because when you find out that people are tired of having to drive around a large bay, and want a quicker way to get across it, you can suggest something like a bridge.  You can explain that it won't damage the cars when they land, and will overall just be a better solution.

Dependency management

Ok so you have your management culture right, and you have good business processes in place, you're all set right?

Wrong.

The last enemy of a software team is the complexity spirit demon.  This topic can get quite technical, so to keep it easy to understand I will use a non-software example.  It basically boils down to good organisation.

Something that is quite well organised would be a supermarket.  You go around the shelves, you fill your trolley, you go and pay.  This has what we can in software, well designed "interfaces".

An interface is just where two things come together and interact.  So when you pay, you have a debit card, they have a card machine, and everything is good.  The interface hides behind it the mechanics of how each problem gets solved.

The supermarket doesn't know how you made your money, which bank you are with or anything else.  And you don't know which payment provider they are using, how they manage their accounting systems, or pay their taxes.

And you don't need to.  So the interface only contains the relevant information, "do you have enough money to pay for this?".  And it returns a yes or no answer.  If it's a yes, the funds are transferred, if it's a no, the goods are returned to the shelf.

It's simple.

It also provides freedom.  You can move to another bank, you can add money to your account through wages, you can spend them elsewhere.  In fact you can do anything you want to with your bank account, as long as you honour the interface - i.e. you have enough cash in there when you do your shopping.

The supermarket has other examples.  You don't know exactly where the eggs came from, which farms, when they are delivered etc.  And you shouldn't need to.  Imagine if going to the supermarket to buy eggs involved checking the shelves to see where the eggs were being laid that day, then having to travel to a different farm each time to collect them.  You wouldn't know if buying eggs would take you 10 minutes or 10 hours.

You as a consumer would then start creating things to deal with this complexity.  We say that irrelevant information has "leaked" out of the interface.  And you are "depend" on information behind the interface.

This means that the supermarket would not easily be able to switch to a foreign supplier, because now every customer wouldn't just need a car to buy eggs, but some form of air transportation.

So badly defined interfaces create complexity, because you cannot easily make the necessary changes without breaking the entire system.

Summary

This type of thinking can be called "Agile Development", or "Lean Development", but honestly oftentimes these just become buzzwords devoid of any real meaning.  A lot of focus can rest on the rituals of these development frameworks, instead of the underlying principles of why they are there.

The first company I mentioned "did agile".  We had daily "stand up meetings" (i.e. progress reports), and used a "kanban", and had "sprints".  But honestly we violated almost every agile principle you can think of.  There was no prototyping, (as I said we barely shipped any actual code before ditching an idea and moving onto something else), no customer feedback (the entire backlog was invented by the product manager and planned out 2 years into the future).

Most of the actual relevant and important ideas behind agile development were ignored and instead we focused on the "agile" rituals.  The result was a total mess.

So by focusing on the main sources of waste, and working to reduce those, you get a development team that is actually productive, with high moral and lower stress, that produces higher quality code that lasts longer.

Ultimately this is a better use of business capital and creates more value for businesses in the longer-term.