Getting Started with Docker

1 Comment

Getting Started with Docker

Docker is a controversial technology. Depending on who you talk to it's either the future of software, available now or a rickety half-baked system which causes nothing but problems. The truth is a little more nuanced. Docker does a lot of things right, but it gives you the freedom to shoot yourself in the foot.

Something everyone can agree on is the difficulty a new Docker user faces when it comes to figuring out best practices. A lot of people keep running into the same problems.

We've been using containers for deployment of Node.js and PHP applications at my agency, Web Artisans, for the last 12 months. Docker has changed the way we ship software, but it hasn't always been smooth sailing. We've made our mistakes, learned from them and this is my effort to put together the key dos and don'ts of running Docker containers in production.

Let's start with a few principles which will make your first production Docker application more manageable.

Containers wrap processes

It's easy to get carried away when building a Docker image. You build an image based on ubuntu:trusty and go nuts with apt-get installing everything your service needs to interact with the outside world.

Before you know it, you've packaged up nginx, php-fpm, redis, memcached and mysql into a massive container image which takes 20 minutes to build and you're wondering what all the hype is about. Wasn't one of the promises of modern web dev a tighter feedback loop than ever before?

Containers should be more granular. You're shipping execution environments for individual processes, not virtual machine images. Have a container for your application, a container for redis and a container for nginx. Containers are lightweight. You don't need to bundle everything into one image.

Ship your application code in a container

Mounting your application code as a volume in a container with the Node or PHP runtime is great for development and making changes on-the-fly, but it's a pain when you go to deploy your code in production. You can't easily take advantage of orchestration benefits like rolling updates or canary releases and it's hard to tell which version of your app a given container is running if you're relying on volumes.

Build and tag your app container with the version contained within and enjoy easy deployments and predictable behaviour.

Keep your images small

One major concern when developing with Docker is managing your library of images. Pull in Ubuntu and you're already looking at a 300-400mb container just to ship 1-2mb of your application code. Account for a few dependencies, a new container build a few times per week and you end up staring down the barrel of a low disk space warning very quickly. Storage aside, it's never fun waiting for a massive image to pull whenever you want to spin up your app on a new machine.

Alpine Linux is a dramatically more lightweight Linux distribution which gained tremendous popularity with Docker users. Alpine's image is a mere 5mb and Alpine-based images can be measured in the tens of megabytes, instead of hundreds. Alpine comes with a full-featured package manager and shell. If you can handle learning some new commands Alpine will put your images on a serious diet.

Most popular official images also offer Alpine-based variants. Check the tags for your favourite images to see if you can slim down your dependencies too.

Containers are disposable

Do not use containers for data persistence in production.

Let me repeat this, because it is critical.

Do not use containers for data persistence in production. If you use containers for data persistence you will have a bad time.

There are a number of reasons for this which I'll delve into in a future article, but essentially it boils down to three things:

  1. Containers are great because they can be quickly spun up or down,
  2. Services which scale horizontally are great candidates for containerization. Services which scale vertically (like databases) are not, and
  3. There can be weird I/O and latency issues with container-to-container networking and volumes which isn't great for database use.

Database containers are great for development. It's awesome being able to spin up a disposable DB container in a CI environment. In production, you should only deploy containers which can safely be terminated and their filesystem wiped at any time, without warning - application containers.

Use an external database server with fail-over. Store files in S3 or Google Cloud Storage. The only data persisted inside a container should be an ephemeral application cache which can be safely lost at any time.

Containerization and Orchestration are different problems

Docker tries to do too much. As a container solution it is perfectly fine, but orchestration of a fleet brings with it a boatload of new challenges. Docker Compose is fine for spinning up a quick-and-dirty development or CI environment, but it's lackluster when it comes to building out production infrastructure and coordinating deployments. Docker Swarm is immature and lacks critical features like rolling updates that you get for free with a (relatively) mature orchestration solution like Kubernetes.

We've been running a few Kubernetes clusters for the past year and we've had a lot of success doing so. The command line management tool is extremely powerful and the declarative manifests make it obvious where all of your configuration values are coming from. The Secrets API also gives you a convenient way to store sensitive data for your application outside source control.

There are other providers of orchestration software (like Amazon Container Service,) however we've had a great experience with Kubernetes and will continue down that path for the time being.

Buy into an ecosystem

It's possible to spin up a few VMs on Digital Ocean and bootstrap your own Docker registry and Kubernetes cluster for your production deployment, but that provides a whole new set of challenges. Networking and container orchestration are the hardest parts of getting a new deployment off the ground.

The Kubernetes project has been working hard to make this process easier in their newest versions, however buying into an established provider and their ecosystem is the quickest way to get your application into production.

Google Container Engine (GKE) is a mature provider of hosted Kubernetes. There are a few other providers out there, but we've found Google's implementation to be the most stable and seamless solution to date.


This is the first in a number of container-related articles I'm planning to write. I'd be interested to hear your experiences using Docker and various orchestration solutions.

1 Comment

Javascript Fatigue is a Choice

1 Comment

Javascript Fatigue is a Choice

It's in vogue to hate the Javascript community right now - especially if you're part of it. If you spend any time on Hacker News you've likely seen a consistent stream of new Javascript frameworks, libraries and build systems cross your screen, each of which attracts tens or hundreds of comments saying "no more!"

You've also seen claims of "Javascript Fatigue" from developers diving into those discussion threads, deriding the "treadmill" that the language and ecosystem has become, complaining of how keeping up is "a struggle."

Even within library-specific enclaves of the Javascript world things are falling apart. Developers were outraged when Angular 2 broke compatibility with Angular 1. Claims of how disruptive migrating to 2.0 would be to teams and development alike were everywhere. Developers felt betrayed after investing years into learning one way of doing things, only to be forced to throw it all away and learn something totally different.

The Javascript world is turning into a fractal of frustration and derision, driven by developer exhaustion.

This is Madness

It's not your (and I assume you're a developer, if you're reading this) job to keep your codebase up-to-date with the latest and greatest Javascript techniques. Your Angular 1.x code didn't suddenly stop working because Facebook launched React. Your Browserify config didn't spontaneously combust because Webpack made #1 on Hacker News. If you have a teamful of people trained on Ember or TypeScript, it doesn't make sense to transition them over to React or Elm.

As a developer, your first duty is to the customer, your second duty is to the product and your third is to your team. Choice of language or library isn't even in the top 3. Use what works, use what you know. It's not going to crumble in your hands just because something new came along.

Opt Out of Javascript Fatigue

Product managers, customers, CxOs, none of them care about the hottest Javascript tech. If yours do, you should start running. If you're exhausted by it, get off the treadmill for a bit. Focus on your chosen stack. Train your team on your chosen libraries. Leave your FOMO at home. Build something amazing.

Afraid of getting left behind?

Don't be.

It'll all be there when you get back.

The Hacker News posts, the Medium essays. Even the posts on <your favourite startup>'s engineering blog.

You might have missed a few libs that blew up, got big and then just as quickly died.

Who cares? You made something. Now you can take a moment to learn about the Next Big Thing... if you want. Or you could just make another cool thing with the stack you know.

Opt out of Javascript fatigue. Opt in to making cool stuff.

1 Comment

Flying Economy is False Economy

Comment

Flying Economy is False Economy

Too many bootstrapping entrepreneurs (particularly the location independent kind) don't spend enough money on themselves. If you're baselining somewhere and currently shouting at me through your computer screen, keep reading - you probably need to hear this.

Last December I made the decision that going forward, I would fly business class whenever I travel internationally. Not as an exercise of ego, not even because of the free champagne on board (though that is definitely a nice perk) - for me it's all about avoiding friction.

Too many people dramatically overestimate their ability to handle friction. The small, sometimes unnoticeable stresses of day to day life erode your mental state, destroying self-discipline and clouding the mind, leaving you unmotivated and uninspired at the end of the day. Many highly successful people recognise this and go to extreme lengths to eliminate any source of friction, however tiny, in their daily routine. Often with amusing results.

First day back after paternity leave. What should I wear?

Posted by Mark Zuckerberg on Monday, 25 January 2016

Imagine crawling through the inevitable long queue during check-in, battling the crowd at security and immigration, trying to block out the wailing of a rabble of kids racing around the gate lounge while exhausted parents try to pretend they're far, far away. Make it through all that and you can look forward to cradling someone's flab in your lap for your three hour flight, while you ponder what on earth is in the slop that they've served you for dinner.

God forbid you have a connection to make at the other end.

Dealing with (and reacting appropriately to) all of these small stresses adds tremendous cognitive load and wears you out. If you travel frequently, this will have a dramatic impact on your performance in business and life.

This morning I had a 6:40 departure out of Saigon. I woke at 5:00 and was in a taxi ten minutes later. I skipped the shower at home to get a few precious extra minutes of shut-eye at home. Had just enough time for breakfast in the lounge before I was called for boarding. Walked straight onto the plane and nodded off in my spacious seat, reclined for takeoff courtesy of a friendly cabin attendant. An extra hour of quality sleep on the lie-flat skybed during cruise meant I arrived in Hong Kong well rested, ready to take on the day. Almost. After a hot shower and shave in the HK lounge I grabbed a fresh coffee and settled down to work in the business centre.

My work setup in the Bridge lounge, Hong Kong.

My work setup in the Bridge lounge, Hong Kong.

Back when I flew economy I wrote off the idea of working on a travel day - and after long days in transit like today my productivity would be down for a day afterwards. Today I was able to run my company effectively, meaning none of my team were waiting on me, burning valuable time. I'm also going to arrive at my eventual destination feeling like a human being, rather than a nervous wreck.

By investing in my own comfort I am investing in the success of my business.

This applies to other parts of your life, too - not just travel.

A comfortable apartment; tasty, nutritious food which you like to eat; an inspiring workplace and the right tools for the job. All of these things reduce daily friction and make you more effective. Everyone has the same number of hours in the day, some people just set themselves up better to make the most of them.

This is why I pay a premium when I travel, and why I insist all new hires at Web Artisans get the best tools money can buy. It's an investment in our collective success. Anything less and you're just crippling yourself.

Comment

2015 In Review

1 Comment

2015 In Review

Last year was my first year jobless since I dropped out of university to pursue gainful employment in 2011. It was a year full of highs, lows and tremendous personal growth, punctuated with business trips all over the world.

January

I started the year back in Australia visiting with family. I was freelancing, building a freight and logistics platform while servicing smaller customers and stress was high. Balancing an almost full-time commitment to the logistics project with other clients and seeing family and friends was a challenge that I spectacularly failed to rise to.

I ended my trip back to Australia with a visit to my favourite whisky bar with an old friend before jumping on a plane back to Vietnam.

February

February I finally decided enough was enough and I pulled the plug on business for a weekend with a trip to Mui Ne/Phan Thiet. A weekend away with a private villa by the sea sans laptop was just what I needed to recharge. I was so intent on staying offline that I don't even have any photos of the destination.

March

In March I started tracking my life with Gyroscope. I took my first trip to Hong Kong courtesy of the freight and logistics startup. A backstage tour of the ocean freight terminal and countless meetings with freight forwarders gave me a new insight into just how complex moving boxes around the world can be. Unfortunately the weather was pretty terrible so I didn't get to see much else of the city.

I did run into a friend whose ship came into port on my last night in town and spent the night catching up and watching his band perform. Serendipity.

Fed up with constant construction noise at my apartment in Saigon and stressed out by the constant madness of the city I decided to head to Japan for a week to de-stress. While I was there I had a little reminder of my temporary home:

April

A rollercoaster of emotions in April as the logistics startup decides to shut down a week after a friend comes over from Australia to join the team. We commiserated over lunch and I hung out my shingle as a full-time freelancer. With a ticket to Hong Kong paid for (and no reason to be working there) what would otherwise have been a business trip turned into an unexpected holiday.

With better weather, I finally got to see the place in all its glory.

The sudden loss of what was a remarkably stable and quite sizeable contract was a sudden shock, but in retrospect I think I handled it well. The day the news was delivered I started reaching out to my network and as a result within a couple of weeks work started flooding in. All in all, things could have been a lot worse.

May

With more clients rolling in I finally decided to get serious about my business and Web Artisans was born. My lease was finally up on the apartment adjacent to a building site, so I moved to one of the outer districts of Saigon in search of some peace and quiet. At the time it seemed like the perfect course of action. In hindsight the 12 month lease was not a smart move.

I also designed and commissioned my first piece of custom furniture. A few hiccups during the process, but ended up with a final product I could be proud of.

June

I was getting restless in Saigon and wanted to get out of town for a while. I took a weekend poker trip to Phnom Penh with a friend and got to take my first tuk tuk ride around town. It was a somewhat luxurious weekend with minimal stress and I've been meaning to go back ever since. For somewhere so close the airfares are quite expensive though - a side effect of there only being one airline on the route.

July

I took a trip to Japan for my 24th birthday and ended up celebrating with a couple of friends who had their party on the same day. The trip was a healthy mix of classy...

...and slightly less so.

Regardless, it was a lot of fun, followed up by a trip to Tokyo to catch up with the crew over there. Japan's countryside is beautiful in the summertime.

August

After all of the birthday travel in July it was finally time to head down and get caught up on work. The only notable thing about August was the number of times I bashed my head on the upper deck of this damn motorbike-park.

September

Continuing the trend of hard work set in August I rented my first office space here in Vietnam. Unfortunately the orientation and dysfunctional blinds meant my desk was impossibly warm in the afternoon, so I took to heading out to a local cafe in the afternoons to work. It was also on one of the most ridiculously congested streets in Saigon.

October

After a month in the new office I was finally ready to hire my first employee. After considerable capex and innumerable no-shows I finally found the right man for the job. During this process I learned how important it is to hire for attitude, rather than aptitude. This lesson has served me well in the months since.

November

The highlight of November was my business trip to Thailand, where I caught up with a customer for a fortnight of on-site work. I also closed a deal I'd been working on for the previous four months. November was a pivotal month in the development of Web Artisans as a company and also a much-needed bit of fun after a few months of focused, intense work.

Towards the end of November the needs of Web Artisans outgrew our sublet desk in a friend's office, so we moved into a new facility at Dreamplex, a brand new co-working space in Saigon.

We got our name on the wall and I finally started putting in place systems and processes to take myself out of the more mundane aspects of the business. I also hired my personal assistant who has been pivotal in freeing me up to run the show more effectively.

December

2015 came to a close in a rather fitting fashion, with a whirlwind tour around Asia. A week in Hong Kong wrapping up some business followed by New Year in Japan was a fitting close to a year which had, in a lot of ways, revolved around those two countries.

The final minutes of 2015 were intense, counting down at what was arguably the biggest party in Tokyo at Club ageHa, right in the middle of Tokyo Bay.


Last year was a year of business. I grew Web Artisans from a one-man show to a real company with four full-time employees and two contractors. That came at a cost though. Work-life balance and personal relationships took a back seat. My personal development (intellectual and physical both) took a back seat and I put myself under a lot of stress. All of this had a big impact on my health and happiness.

2016 is going to be a year of focus on myself. Developing healthy habits, reading more and doing more things I enjoy. Particularly writing and skiing. So far I've started the year by leaving my laptop at the office every night and being out of there by 6pm at the latest. I'm aiming to read two books per month and review at least one here on my blog. I'm also going to sleep and waking up earlier so that I can properly plan my day.

I'm looking forward to sharing more of my strategies for getting my life in order over the next few months. Thanks to all of my friends and family for their constant support and I'll see you all throughout this coming year!

1 Comment

Bad UX Breakdown: Airbnb

1 Comment

Bad UX Breakdown: Airbnb

Bad user experiences are all around us, and there's no worse time to run headlong into them than while travelling. Deadlines are tight, there are a million and one things to do and it's very easy to become distressed at a time when you're meant to be de-stressed. Today I was on the receiving end of it on Airbnb's website and enough was finally enough. It's time for an in-depth analysis of a UX disaster.

Setting the Scene

I had just woken up on the last day of my stay at one Airbnb that a friend operates. I had to be out that morning in time for the place to be cleaned for an arriving guest. I was trying to book another place I'd been pre-approved for when I ran into a screen which cheerfully announced that new verification requirements had been put in place and that I'd have to submit a bunch of information and confirm my email address.

There was also a somewhat sinister sentence tacked on below the main body copy.

"Your booking will not be placed until you complete verification."

Talk about instant stress. There's no guarantee that I won't lose my booking if someone swoops in before I finish this verification process? Cue panic while I scramble for my passport.

Ticket booking companies have been getting this one right for years now - it's an easy fix. Let the user know that you're holding their booking for the next 10 minutes while they complete the process - it instantly defuses the situation and gives the user some breathing room.

I thought my dramas were over after submitting my passport and getting it verified, but they were only just beginning.

"Please verify your email address."

Immediately after the passport page I landed on a page asking me to verify the email address on my account. There was something odd about this. I've verified my email address before. I shrugged and checked my inbox, desperate to get through the ordeal ASAP. No confirmation email. Hammering the "resend" link did nothing to solve my problem. Checking my profile page I noticed the email address they were using was the same one that was linked to my Facebook account - an email address that has been inaccessible for years now.

There was also a puzzling UI element which left me perplexed.

See that little pink padlock? Possibly the most aggravating little bastard in this whole saga.

See that little pink padlock? Possibly the most aggravating little bastard in this whole saga.

I looked at all of these inputs on my iPad and saw a bunch of tacky locks off to the side. Was this data locked? Would I be able to change it? I tapped on the field, hoping for the best. I got a cursor and the keyboard popped up! Success, or so I thought.

I entered my new email address and was prompted to head on over to Facebook to confirm my password. All well and good, I entered my password and was bounced back to Airbnb, where this absolute gem of an error message awaited me.

There were no validation errors noted against the form inputs, no indication of what went wrong and as I sat there puzzled, the error shown above promptly vanished.

Everything I thought I knew about the profile page was cast into doubt in that instant. The padlock icon sat there off to the left of the email field, staring me down, eating away at everything I thought I thought I knew about <input type="text"/>.

I had no idea what to do. Thinking it might have been a temporary random error, I tried again. Same result.

Googling for the error turned up nothing.

At this point I was seriously concerned about someone booking my apartment out from under me. I even considered creating a new email address and a second Airbnb account.

All of this angst brought on by one simple problem. A terrible, pathetic excuse for an error message.

The Secret to Writing Effective Error Messages

It's super simple. Tell your user what went wrong. If you can't do that, tell them where to get help.

In this case, I had no idea whether there was a system problem on Airbnb's end, if my Facebook account had a problem, if the email address field was indeed locked (I discovered later on my laptop that if you hover over the padlock it indicates privacy, but there's no hover state on a tablet) or if there was some other issue at play.

I also didn't know what the next steps were.

This should have been where I was directed to a support contact form to get immediate assistance. Instead, I was left with nothing but my wits - and I was at my wits' end.

When all else fails, try anything and everything.

Out of desperation I tried the first thing that came to mind. Thinking the email address might be locked to my Facebook address I went to change my email on Facebook.

Little intermission for a special extra bonus fail courtesy of Facebook:

Facebook developers: why not just do what developers for years have done and send a bloody email to the address with a confirmation link that I can click on? I've been receiving email at that email address for years now.

At this point I was just about ready to throw my iPad across the room. Thankfully I identified that particular course of action as a bad move and decided against it.

For whatever reason Facebook accepted my work email, so that's nice. Heading back to Airbnb I tried setting my email address to the new one I'd just set up on Facebook - and by some miracle it actually worked. I was able to confirm my booking and I was on my way to having a cozy bed for the night.

So, does anyone care to guess what the constraint on the email field was?

Astute readers might think that it has to match the Facebook email address for linked accounts.

Sorry astute readers, you're wrong. I thought the same thing at first - it fits the evidence, right?

Yes and no. I dug a little deeper. I thought it odd that I had previously confirmed my email address on Airbnb at my personal address and yet I was being asked to do it again at an email address I hadn't used in years.

It turns out, I had two Airbnb accounts and didn't realise it.

I had previously set up an account which wasn't linked to Facebook. As part of their standard onboarding process Airbnb prompts you to connect your social accounts for verification and reputation purposes. I did this via their dashboard and everything looked fine on the surface. For whatever reason though (possibly because the email address I used on Airbnb didn't match my Facebook email address, at a guess?) a new account was created instead of simply linking my Facebook profile to my existing Airbnb account.

That pathetic blue error message noted previously should have read "email address already exists."

Postmortem

There are a few key lessons to be learnt here.

  1. Test thoroughly. Especially account management tools and external integrations.
    I now have my reputation and reviews split across two Airbnb accounts. The only way I can use my personal email address on my personal account is to delete my original account. All of this could have been solved by a smarter, more thoroughly tested Facebook integration.
  2. Write better error messages. Reach out to users affected by unknown/unexpected errors.
    When there's an error, tell the user what they did wrong. If you don't know what they did wrong, direct them to a contact form which puts them in touch with someone technical. Even better, follow up with them directly and work with them to identify the flaw.
  3. Don't be too clever.
    I'd be willing to bet that the reason Facebook rejects my email address is the fact that it starts with admin@ - if they just used the classic tactic of pushing out a confirmation email I'd have been able to use my current email address and none of this would have happened.

Keep these lessons in mind when building your next application. Let's work together to make the web a better, more usable place.

1 Comment

The High Price of Cheap

4 Comments

The High Price of Cheap

Cut-price code cutters on Elance, oDesk and other outsourcing marketplaces can be an attractive prospect to any business owner, especially so if you're a bootstrapped entrepreneur. Unfortunately many people shopping around on these sites are blissfully unaware of the high costs hidden behind the low price tag.

In the last month alone, I've helped rescue two projects which were on the verge of disaster. One had the developer vanish off the face of the earth overnight with 20% of the project unfinished, the other was simply a result of lack of skill and integrity of craft. Looking at the last year, the total body count is closer to eight.

Outsourcing isn't all bad - in the right hands for the right tasks it is a powerful tool - but we'll get to that bit later. First a case study on the garden-variety outsourcing disaster, the same one I've seen unfold many times before.

The Perfect Storm - An Outsourcing Disaster

Our business is about to undertake an ambitious software project and a project manager is being appointed. Nobody knows a business like the people on the inside, so the best man for the job must be an employee. Unfortunately no one on staff is an experienced developer - but that's a moot point; it's why we're outsourcing. They'll take care of that for us. Proposals are vetted, calls are made and meetings are scheduled.

The applicants look good. They have nice people with glowing quotes plastered on their home page. Their portfolio looks good, they have all of the right answers, they're confident they can deliver and they fit the budget. Contracts are signed.

As the project commences things power along. The marketing machine goes into overdrive and the company shouts news of their up-and-coming software from the rooftops. The mailing list laps it up, eagerly awaiting the launch of the product.

Meanwhile, behind closed doors, things aren't so rosy. Version one lands and it's not quite right. Internal errors aren't being handled and users are seeing ugly error pages covered in obscure technical references, some "obvious" things are apparently not so obvious, in extreme cases functionality is simply broken. Change requests are lodged and lists are made. First one deadline is missed, then another. Meetings are rescheduled, assurances are made. Terms are renegotiated. The project grinds on.

Launch day approaches. Two weeks out, the to-do lists are a mile long and spread out across dozens of Excel spreadsheets bearing names like "TODO-Fixes v2 final Copy - 3.xlsx." Changes that should be straightforward are taking hours, or even days to implement. Bugs are popping up faster than they can be squashed. The project manager is tearing his hair out, the outsourced developers are more interested in dodging blame than fixing problems and the rest of the parties involved are shaking their heads, wondering how the project ever got into this mess.

This is usually where I get called in.

The Software Problem

The reason so many outsourced software projects fail is a simple one. Software is opaque to the non-technical user. Sometimes a user may catch a glimpse of the messy internals via a poorly-handled error which may dump the ugly guts of a program to the screen, but for the most part users only see what goes on on the surface, with no idea of what lurks beneath. A faulty system built on shaky foundations can look perfectly fine, or even desirable to the casual observer - as long as a pretty user interface has been slapped on top.

city.jpg

In projects like these the foundation is invariably a rickety jumble of code, held together with the digital equivalent of duct tape and chewing gum, threatening to fall apart at the lightest touch. An undocumented, unpredictable pile of crap which takes even an experienced software engineer days to sift through and comprehend.

Occasionally there's even code copied and pasted from tutorial websites, with the references to said sites intact.

Software development is one of the few industries in which it is easy to polish a turd.

Using cheap labour for your software project is not viable if you can't audit the work produced by your team. If you can't hold your developer accountable from a position of knowledge you're on track for disaster.

The High Price of Cheap

By the time a client has engaged my services, typically their outsourcing team has been hard at work for weeks, or even months. They have produced a system that may look acceptable from the outside, but internally requires a massive investment of time to get to a maintainable, stable, secure state.

A change that may have taken an hour in a professionally architected application can take up to ten times as long in a system riddled with half-baked shortcuts and copy-paste code.

If changes are expensive - and they are - the cost of maintaining the product over its lifetime is an order of magnitude higher. New features developed in the future cost many times what they should, simply because of the overhead involved in testing such a system thoroughly to ensure nothing critical has failed.

A small saving up-front leads to massive expense down the track.

Effective Outsourcing

It's not viable for every business owner to learn to sling code, so we're left with the obvious dilemma - how does a non-technical business owner outsource effectively?

If you have a friend who is an established developer, engaging them to vet outsourcers is the ideal solution. Even better if they can check in throughout the whole project to ensure things are still on track.

If that's not an option I have developed a set of software project checklists for evaluating your prospective freelancer or outsourcing agency before your project gets off the ground, after takeoff and when your shiny new software lands in your inbox upon completion.


Software Project Preflight Checklist

ALWAYS perform these checks before TAKEOFF and when reviewing an OUTSOURCING PROPOSAL.

  • Are there clearly-defined deliverables and delivery deadlines specified in the proposal?
    It's a well-known phenomenon in the industry that software projects expand to fill available time, so fixed deadlines are essential.

  • Will you have access to a staging or preview environment at all times to monitor the progress of the project as it is developed?
    In the case of desktop applications, are there dates scheduled for live demonstrations or delivery of preliminary versions of the software?

    Pre-recorded screencasts or scripted demos are not acceptable. Insist on being permitted to use the application yourself.

  • Are the services being provided clearly and specifically defined?
    Beware of fluff words such as "optimisation," "clean-up" and "feature development." These are often used as catch-all billable items when a project has fallen far behind schedule and costs need to be recovered.

  • Are there terms permitting you to terminate the contract in the event of missed deliverables/deadlines or unresponsiveness in communication on the part of the agency or freelancer?
    It is unfortunately common for emails to languish for days or weeks, or even for developers to completely vanish at the hardest, most important stage of a project.

  • Is the agency or freelancer actively trying to learn about your business and your requirements?
    If they aren't asking questions up-front you should be prepared for a lot of back-and-forth later in the project lifecycle.

Software Project In-Flight Checklist

Perform these checks REGULARLY during NORMAL FLIGHT and IMMEDIATELY upon encountering UNEXPECTED TURBULENCE.

  • Is the project on-track to meet its next deliverable/deadline?
    This is hard to judge as a non-technical person, but trust your gut. If screens are missing, or errors are appearing which aren't being fixed, say something. In web projects especially, if there's no visible progress for a few days you should be concerned.

  • Are your questions being answered to your satisfaction?
    Look out for non-answers, or stonewalling by agencies. A common answer is "I'm waiting to hear from Steve who knows more about your project." If you hear this and do not get to email Steve directly chances are your outsourcer is also outsourcing your work, which is a big red flag. Vague answers should also make you wary - demand specifics and don't tolerate fluff.

  • Is there attention to detail in the user-facing parts of the application?
    Obscure error messages dumped to the screen, designs that don't look exactly like what were provided, inconsistent typography and web pages that are not responsive or take a long time to load - all of these things are indicative of a team lacking attention to detail. If flaws are visible externally the internals are most likely ten times worse.

Software Project Landing Checklist

ALWAYS perform these checks BEFORE FINAL APPROACH and when TAKING DELIVERY of your software.

  • Is the final production software installed on a server you control?
    Many agencies or freelancers offer cheap hosting or simply don't bother to install the software on an independent server. This results in an uncomfortable conversation (and the occasional lawsuit) in the event that client and developer decide to part ways in the future. Always insist that the software be installed on a third party's system, or your own dedicated server.

  • Do you have an editable version of the software (the source code)?
    Always ensure that your developer provides you with the source code for your software. That gives you the flexibility to enlist other developers in the future. Additionally, if your freelancer disappears, you can get someone else to pick up where they left off.

  • Have all agreed-upon deliverables and outstanding tasks been completed?
    Do not pay the balance on your project until you have received all deliverables as per the previous points - feet drag to an incredible degree after money has been received.

Following these checklists will not guarantee a pain-free outsourcing experience, but it will dramatically improve your odds of success. Ultimately what decides the fate of your project is your judgement in selecting a developer or agency to work with. If your business runs on software it is an investment, not a cost. Treat it like one and do not compromise on quality.

4 Comments