Making Software - a three step process
April 14, 2020
One of the most useful tips that has guided much of my decision over the years has been this simple principle: three steps, executed in sequential order;
- Make it work
- Make it right
- Make it fast
These steps outline the process through which software should be made. You should refer back to these steps and discover for yourself which step you are currently in while creating software. This will also help you identify whether you need to or indeed can move to the next step.
An important distinction to make up-front is that you don't necessarily need to complete all steps in order to ship software to users. Let me explain.
Make it work
The first step should be the most obvious, but the key to (and the most difficult) is acknowledging you are at step 1. Making software work is about glueing all of the pieces together until the thing you're trying to build actually works.
Imagine you're building a skateboard and you take a plank of wood and screw 4 wheels to the bottom. There should be criteria to allow you to recognise when it's finished and can move to the next step, for example:
- Can you stand on it?
- Does it move forward when you push off?
If the answer to both of these questions is yes, then congratulations you've got a working skateboard.
Keen readers will poke holes in the analogy - the skateboard will break easily because the wheels aren't correctly fastened to the plank and will buckle after the first ride. While this may be true, the first step is making it work and we may not be expecting to produce thousands of these skateboards and allow customers to purchase them at this stage.
Bringing this back to software, if you catch yourself on step 1 and you're already thinking about how to optimise for performance or you've got the best abstraction idea that is extensible to the nth degree, then the battle may very well be lost because if you can't make it work, nothing else matters.
This step is more about what you don't do straight away, as opposed to what you do.
Make it right
Continuing the skateboard analogy, this next step provides you with the time built into the process to take a step (pardon the pun) back and look at the big picture. Such as, the wheels need to be fitted correctly with the proper materials and safety considerations to withstand the rough and tumble expected in the riding of a skateboard.
This is no different to the wear and tear of software - without the proper guards in place such as tests, abstractions and extensibility in place the software will likely buckle under the pressure of real users.
This step is the right time to take the working thing and build it properly, regardless of whether you start it from scratch. The first step is more discovery than anything else and provides you with the confidence and knowledge of how to build the thing, so that in this step you can build the thing right.
Making software right ensures you have the correct checks and balances in place, such as applying principles to common problems and giving it the best chance of long-term sustainability.
At each step, a decision needs to be made about whether to progress to the next - for example there might be times in building software when making something work is more important than making it right. The key difference between this and pure negligence is the fact that this trade-off is a conscious choice, which makes it hard to see in hindsight unless well documented. It is often seen as tech debt, accrued for a purpose, but don't be fooled into thinking it's at all similar to financial debt (a common misconception, that may be for another time).
While I wouldn't recommend stopping before making it right, there are times where making something is better than not making anything at all. The hardest part is accepting the snowball effect this will have later down the track and whether you are actually prepared for the true cost.
Making it right, while technically a choice (hence it being a different step) is probably where most early software needs to land to be effective. Going further than this may become detrimental unless you have a reason to make it fast.
Make it fast
The final step is where all guns are blazing, software is often in production and you need the last piece of the puzzle to ensure the software works as intended (i.e. fast enough for the user). This step is all about optimising what you have done in the previous two. To keep the skateboard analogy going, at this step we would want to focus on the speed of the rider ensuring they maximise speed.
In software on the other hand, this may only happen once it's actually in the hands of users, so you know where the bottlenecks are. Sure, you could throw more hardware at it temporarily but eventually you have to make the software fast enough to scale correctly and appropriately.
In each step there are trade-offs but in this step the smallest decision can have the biggest impact, so you should always make data-driven decisions based on real usage rather than making educated guesses and hoping for the best. In time, experience will often tell you where the bottlenecks are, but this isn't always the case and depends on the system. Making it fast requires skill and practice and sometimes you don't even need this step until the software becomes popular enough that you have a reason to focus on speed. This is why it's best to analyse the data before embarking on this step.
One step at a time
In each step there is particular nuance to doing it right and doing it well. It's not like a recipe where the instructions tell you exactly what's required. All you can do is take it one step at a time and figure out where you can draw the lines around it. At times it may even be harder than building the software on it's own, but being able to do this consistently will help you in the long-run.
I have found great focus from following these three steps when building any software professionally.
February 29, 2024
A common question in software engineering interviews is how can you speed up a slow query? In this post I want to explain one answer to this question, which is: to add an index to the table the query is performed on.
February 12, 2024
I spend most of my time thinking about performance improvements. Refactoring is tricky work, even more so when you’re unfamiliar with the feature or part of the codebase.
May 31, 2023
Asynchronous programming is a foundational building block for scaling web applications due to the increasing need to do more in each web request. A typical example of this is sending an email as part of a request.
April 01, 2022
I have mixed feelings about feature flags. They are part of the product development workflow and you would be hard pressed to find a product engineering team that doesn’t use them. Gone are the days of either shipping and hoping the code will work first time or testing the life out of a feature so much that it delays the project.
March 18, 2022
When I first started interviewing candidates for engineering roles, I was very nervous. The process can be quite daunting as both an interviewer and interviewee. The goal for the interviewer is to assess the candidate for their technical capabilities and make a judgement on whether you think they should move to the next round (there’s always a next round). Making a judgement on someone after an hour, sometimes a bit longer, is hard and error prone.
June 03, 2020
Dependency Injection is the method of passing objects to another (usually during instantiation) to invert the dependency created when you use an object. A Container is often used as a collection of the objects used in your system, to achieve separation between usage and instantiation.
April 17, 2020
Working from home has been thrust upon those lucky enough to still have a job. Many aren’t sure how to cope, some are trying to find ways to help them through the day. Make no mistake, this is not a normal remote working environment we find ourselves in, but nonetheless we should find ways to embrace it.
April 14, 2020
One of the most useful tips that has guided much of my decision over the years has been this simple principle: three steps, executed in sequential order;
October 24, 2019
Code Reviews are one of the easiest ways to help your team-mates. There are a number of benefits for both the reviewer and pull request author:
September 12, 2019
It’s been a while since I last wrote about why testing is important, but in this post I thought I would expand on that and talk about why not only unit testing is important, but how a full spectrum of automated tests can improve productivity, increase confidence pushing code and help keep users happy.
July 05, 2019
Design Patterns allow you to create abstractions that decouple sections of a codebase with the purpose of making a change to the code later a much easier process.
May 03, 2019
Umbrella apps are big projects that contain multiple mix projects. Using umbrella apps feels more like getting poked in the eye from an actual umbrella.
April 14, 2019
Ever get the feeling that adding this "one little hack", a couple of lines of code, won't have much of an impact on the rest of the codebase? You think nothing of it and add it, convincing your team members it was the correct decision to get this new feature over the line. In theory, and generally speaking, I would kind of agree with doing it, but every hack is different so it's hard to paint them all with the same brush. If you've been doing software development for long enough you can see this kind of code coming from a mile away. It's the kind of code that can haunt your dreams if you're not careful.
March 04, 2019
Last week was Lonestar ElixirConf 2019 held in Austin, Texas. The conference ran over 2 days and was the first Elixir conference I had been to.
February 01, 2019
In most cases I have found inter-process communication to be an unnecessary overhead for the work I have been doing. Although Elixir is known for this (along with Erlang), it really depends on what you’re trying to achieve and processes shouldn’t be spawned just for the fun of it. I have recently come across a scenario where I thought having a separate process be responsible for performing concurrent and asynchronous jobs would be the best way to approach the problem. In this article I will explain the problem and the solution.
December 19, 2018
When we think about what an application does, it's typical to think of how it behaves in context of its dependencies. For example, we could say a ficticious application sync's data with a third-party CRM.
November 20, 2018
When you're browsing your way through Elixir documentation or reading blog posts (like this one), there's no doubt you'll come across a GenServer. It is perhaps one of the most overused modules in the Elixir standard library, simply because it's a good teaching tool for abstractions around processes. It can be confusing though, to know when to reach for your friendly, neighbourhood GenServer.
October 30, 2018
Typically in an application with a database, you might have more records than you can fit on a page or in a single result set from a query. When you or your users want to retrieve the next page of results, two common options for paginating data include:
September 26, 2018
Protocols are a way to implement polymorphism in Elixir. We can use it to apply a function to multiple object types or structured data types, which are specific to the object itself. There are two steps; defining a protocol in the form of function(s), and one or many implementations for that protocol.
August 23, 2018
Recently, I've been writing a tonne of Elixir code, some Phoenix websites and a few other small Elixir applications. One thing that was bugging me every time I would create a new project is that I would want to add Docker to it either straight away because I knew there would be a dependency on Redis or Postgres etc, or halfway through a project and it would really slow down the speed at which I could hack something together.
July 26, 2018
While writing Understanding Concurrency in Elixir I started to grasp processes more than I have before. Working with them more closely has strengthened the concepts in my own mind.
July 14, 2018
Concurrency in Elixir is a big selling point for the language, but what does it really mean for the code that we write in Elixir? It all comes down to Processes. Thanks to the Erlang Virtual Machine, upon which Elixir is built, we can create process threads that aren't actual processes on your machine, but in the Erlang VM. This means that in an Elixir application we can create thousands of Erlang processes without the application skipping a beat.
July 06, 2018
Ecto is an Elixir library, which allows you to define schemas that map to database tables. It's a super light weight ORM, (Object-Relational Mapper) that allows you to define structs to represent data.
June 27, 2018
We often think about Streaming as being the way we watch multimedia content such as video/audio. We press play and the content is bufferred and starts sending data over the wire. The client receiving the data will handle those packets and show the content, while at the same time requesting more data. Streaming has allowed us to consume large media content types such as tv shows or movies over the internet.
June 06, 2018
A Queue is a collection data structure, which uses the FIFO (First In, First Out) method. This means that when you add items to a queue, often called enqueuing, the item takes its place at the end of the queue. When you dequeue an item, we remove the item from the front of the queue.
March 23, 2018
Elixir is a functional language, so it’s no surprise that one of the main building blocks of the request-response cycle is the humble Plug. A Plug will take connection struct (see Plug.Conn) and return a new struct of the same type. It is this concept that allows you to join multiple plugs together, each with their own transformation on a Conn struct.
February 06, 2018
A Supervision Tree in Elixir has quite a number of parallels to how developers using React think about a component tree. In this article I will attempt to describe parallel concepts between the two - and if you've used React and are interested in functional programming, it might prompt you to take a look at Elixir.
December 21, 2017
Technical debt is a potentially crippling disease that can take over your codebase without much warning. One day, you’re building features, the next, you struggle to untangle the mess you (or maybe your team) has created.
August 15, 2017
Before being introduced to Elixir, a functional programming language built on top of Erlang, I had no idea what pattern matching was. Hopefully, by the end of this article you will have at least a rudimentary understanding of how awesome it is.
January 06, 2017
Elixir is a functional programming language based on Erlang. I’m told it’s very similar to Ruby, with a few tweaks and improvements to the developer experience and language syntax.
November 29, 2016
Unit testing can sometimes be a tricky subject no matter what language you’re writing in. There’s a few reasons for this:
