Why bigger isn’t always better…. Maximising returns through cloud-based proof of concepts:

Blue Down Arrow

Part 1 – Starting out and getting over the first hurdle! 

So, about ALL of that data you have 

Pushing data into the cloud was once seen as a skill confined to tech companies, who had either started with at least one foot in the door or had the skills readily available to make it happen. However,  it is now much more commonplace for organisations to want to lift and shift their data and operations into the cloud, with the promise of releasing both savings and efficiency gains. 

This move to cloudcentric data usage is something that Oakland fully endorses; it is what comes next that is often the real issue. Moving operations to the cloud is a timeconsuming and expensive job, affording it the luxury of its own owner and associated strategy. However, whilst a good strategy may be in place for getting your data away from onpremise systems, a question that will likely stump many companies is what do you plan to do with afterward? 

Shifting an organisation onto a cloud provider isnt cheap, which explains why most of Amazons operating revenue is driven by AWS. As such, its often tempting to offset this cost with large projects, similar in many ways to fabled ERP implementations. However, with the promise of significant rewards comes significant risk, and for many companies, this is an unknown area that can often lead to a whole new set of problems. We’ve certainly seen evidence of these issues working with our clients. With sprawling new data architectures designed to link core processes, development teams locked into long projects with little flexibility and struggling with ways of working that inhibit more than they improve. 

And that is why we have a slightly different approach to recommend, something that takes the focus off largescale returns and instead looks to build an organisations capability to deliver rapid solutions that prove their valueThroughout this and the following article, Ill be delving into the world of cloud-based proof of concepts (POCs). These projects combine discrete process elements with the myriad of cloud technologies in order to build out internal capabilities, ways of working, and business investment. They typically run over a much shorter development timeframe with high amounts of business input and a focus on achieving the most value from minimum change, i.e., not chasing the edge cases!  

Through these articles, well be able to highlight the successes that weve seen first-hand from this methodology, drawing on real-world examples from the past twelve months. 

Where to start on your POC journey? 

So, we’ll assume that youre on board with the idea; where would you begin to look for something to turn into a proof of concept, and how can you help make it as successful as possible? Luckily Oakland has lots of experience hereso this is our advice. 

  1. It doesnt have to be something brand new – Developing a POC doesnt always mean you have to build from the ground up; changing how something is done, or its core technology is just as valid and fruitful. 
  2. Pick an area that will have strong buyin, either through interest or the value it may drive. This also doesnt have to be a purely tech area either; updating a process related to, say, data governance or improvement cycles using new technology can be a great way to make incremental gains.  
  3. Ensure that your POC has an owner, someone to drive it forwards and make decisions. This is key when moving it into production and beyond, both in terms of features and budget! 
  4. Consider access to data and environments (does a sandbox exist, or does one need creating?). These can be longterm blockers, especially if an external company is doing the work. This may impact which project you choose. 
  5. Factor in the long-term costs of the project, not just the cost to get a minimum viable product; otherwise, you can easily fall at the first hurdle when you try to productionise your POC. 

These may seem rather obvious tips at first glance. Still, weve certainly learned the hard way that its not always the case, with the most common issues being the lack of a strong POC owner and an understanding from the organisation of what it will take to get the finished POC into production.  

Here are a couple of examples that give you a ballpark idea of the costs you could be looking at. 

POC 1 – Converting a process from a proprietary piece of licensed software to work in a more understood and maintainable manner, using opensource tooling and the clients cloud environment. The process itself encompassed around £300m in costs per annum and, while being fairly critical, wouldnt be too perturbed by delays in development. The initial work was carried out with a £50k budget to test what was possible, with an expected £200k cost from start to finish. 

POC 2 – Taking an indemand, dataheavy, excel based analysis and developing it into a scalable cloud application. The results of this analysis were an in-demand product, but users wanted a better way to access the information and add additional features. The initial work was completed for roughly £80kwith a total cost of £130k to bring it into production. 

With these examples in mind, it should be possible to put together a shortlist of your viable options, each with assigned owners, stakeholders, and budgets, from which you can pick as many as required to go into initial development. 

Time to get moving?! 

With at least one project to kick off into development, the work can finally start, right? Its time for our next piece of advice; dont skimp on the planning! This doesnt just relate to planning how the work will be split (though always helpful), but other elements such as: 

  • When will stakeholders be updated? 
  • What are the musthaves and stretch goals of the POC? 
  • What decision points are needed and when? 
  • What are the criteria needed to be worth productionising? 
  • Who is needed at core meetings to keep information flowing? 
  • What governance points are required, and what do we need to cover these? 
  • What team mix do we need to prevent blockers? 

Once you have this information adequately documented and available to share, youre in a perfect place to start. This might eat into your initial development time, but thats better than losing a week because you cant get a key person to explain how something works, or your decisionmaker is on leave! Armed with the plan, the next steps are to start regularly communicating with your internal team and start laying the foundations. This will enable the team to develop something with real potential. Try and follow the 80/20 rule (80% value for 20% effort) and donget hung up on the edge cases that can be filled in later; concentrate on getting the core concept working. 

Wrap up 

So, weve covered the initial process to get started on the POC journey, including a few of our examples and experiences to try and mitigate any of the pitfalls that are associated with development projects! As such, this seems an excellent place to wrap up for now, with my next article going over getting past the cutoff point and where things can fall over moving into production and beyond. 

Richard Louden is a senior data engineer at Oakland

Previous Post: The Use Case For Project AnalyticsNext Post: How to align a data governance and data strategy roadmap