Robust, Practical A/B-testing and the Schedule IT Story
A report from the May 2015 meetup. Entrepreneurs, just like businesses, come in all shapes and sizes but the interesting thing is what they look like means nothing. Every business manufactures a veneer. Not just its brand but everything about its operation from grand offices to private meetings with partners and suppliers. One of the nice things about Lean Startup Yorkshire is the candid way it’s speakers often share the intimate details of their business journey for the benefit and education of the community. This may sometimes be cathartic but it always fascinating.
May’s Lean Startup Yorkshire kicked off with Mark Balance, the founder of Schedule it, performing a run through of his product followed by some frank and insightful history about the company origins and it’s remarkable growth. After starting to build what would become Schedule IT one weekend back in 2004 the plan was never to create a commercial product; but to scratch an itch Mark had himself and create a simple solution to his own scheduling needs at work. At the time he was still a full time employee. After a few years this weekend project had turned into an income generating hobby as Mark had by now attracted a number of licence paying customers. However the tuning point came when his employer announced plans to relocate. The prospect of moving his family to follow his job didn’t appeal and he chose instead to make his little venture a full time business. That was back in 2009 and since then, through a lot of hard work, the trajectory has been skywards.
Now with thousands of users around the world and a string of significant corporate customers Mark reflected on some of the things that helped that sustain that growth. Obviously a full time focus was a game changer but a few other things were called out by Mark. Marketing came up several times and Schedule it’s website has gone through many iterations to establish the brand and importantly to shift the focus from features to benefits and target specific markets who are heavy users of scheduling software. Today the Schedule it website contains around twenty sector specific landing pages that significantly help customers understand how they can solve their problems with the software. Customer response times also came up as a critical factor in wining business and Mark gives this aspect a lot of attention. One things that emerged after launching a mobile version of the software was that giving it a way free was a lot more successful than charging for it simply because it has served as yet another way for customers to discover Schedule it and try it out. Many people discover the free Schedule it in the app store go on to be fully licensed users. This was only discovered after using existing code to build a mobile MVP and experimenting with the exact name and search terms in true lean startup style. The last piece of advice was to set goals and commit to reaching them. Mark reflected on his success when he had done this and was hopping to do it more in the future.
Ethar Alali the founder of Manchester based consultancy Axelisys filled the evening’s main speaking slot. Axelisys help their clients take a more quantitative approach to decision making around product development and marketing through the use of techniques like AB or multivariate testing. Though many people have some knowledge and perhaps experience of running these kinds of tests Ether was keen to emphasis the need for a certain level of scientific rigur. He covered the scientific community’s preference for randomised, double blind controlled trials as the benchmark of objectivity and drilled into practical advice about being sure you know the question your hypothesis is trying to ask and getting to know your covariants plus working to remove, or at least try to control for them.
There was a brief discussion on tools and more questions from the audience on the topic with Ethar expressing advice to keep it simple and if in doubt just start with a spreadsheet. The discussion moved onto multivariate testing’s place within the whole product development lifecycle and its relationship with the various forms of UX testing. The questions rounded off with the topic of testing within early stage startups and when to take a DIY approach and at what stage more expertise may help if funding allowed.