When a recent Thorogood client made the decision to reinvent its approach to information technology, its goal was to become an organization that could adapt its infrastructure to meet changing business needs in as close to real time as possible.
After years of relying on its on-premises architecture to support its data analysis demands, it knew that it needed to transition to a cloud-based environment to better compete in the current market. With the help of Thorogood, the company took that step, moving its business intelligence stack to the AWS cloud. In addition to providing the client with a platform that was better-equipped to meet its needs, the move offered a good look at the benefits that a firm can realize through the adoption of an Agile mindset and a knowledgeable employment of DevOps tools and methodologies. With the help of Thorogood, the company used DevOps to orchestrate a seamless migration and create a tool that is now playing a central role in its data-driven decision in the insurance marketplace.
The Power of DevOps
The challenge of converting legacy systems to a cloud-based architecture can be a daunting one. The move demands a deliberate, thoughtful approach in order to minimize costs and work-flow disruptions, and maximize the cloud’s business benefits. At the client in question, DevOps plays a crucial role in reconciling this tension. The four-pronged framework aims to help IT professionals streamline development and deployment by increasing automation and eliminating redundancy, all while maintaining rigorous standards.
DevOps played a central role in facilitating the client’s recent migration to the cloud, which the company viewed as a big step toward achieving the analytic framework it envisioned. Its overarching goal was to give its scientists the freedom to pursue business insights using self-service tools and the ability to incorporate those insights into future analyses. In order to achieve this perpetual loop of data analysis, it knew it needed to create an environment that was capable of both supporting a variety of applications and easily adapting to users’ evolving needs. Thorogood helped the client create a blueprint for a new cloud-based architecture that utilized multiple platform-as-a-service and infrastructure-as-a-service relational and analytic databases to load and transfer data from a varied set of source systems, creating a data layer that would support its initiatives.
Infrastructure as Code
The as-a-service model is one of the tenets of the DevOps methodology. By approaching infrastructure from a code-based perspective, developers are able to achieve levels of automation, consistency and repeatability that simple are not possible in manual builds. An infrastructure-as-code approach allows developers to use a single file to spin up similar services, which enables them to develop templates with detailed specifications of each service. Within these scripts, they can include parameters that can be dynamically varied according to need (for example, a production resource might require a higher level of performance than a non-production resource).
With its BI migration, our client needed to extract a diverse set of source databases that supported a variety of applications. It also needed to standardize and store that data in the AWS cloud, and then build an analytical data model capable of feeding the applications. Using AWS’s CloudFormation Tool, Thorogood developers created a series of scripts that included specifications of every configuration of every server, which allowed them to automate a number of processes and thereby remove the possibility of human error. With the ability to spin up environments in a fraction of the usual time, projects can get started faster while maintaining compliance with development standards, which are designed to enable auditing and traceability and thence result in better governance. The output is a level of consistency across environments that allows for repeatability and automation.
Version Control and CI/CD
Another core DevOps principle is Version Control, which utilizes a central code repository to maintain all application code files and facilitates collaboration within teams. With the current project, Thorogood utilized Bit Bucket to manage its infrastructure scripts and Azure DevOps Repos to manage its application scripts. Both programs allow for code to be quickly managed in a rigorous environment, giving developers the ability to track changes and restore previous versions of code, reducing the chance of time-consuming mishaps.
Once these scripts are written, or whenever they are changed, they need to be deployed and integrated into an environment in the most efficient way possible, a concept that DevOps refers to as Continuous Integration/Continuous Deployment. With this project in question, Thorogood developers used Bamboo and Azure DevOps Pipelines to build pipelines that deployed infrastructure and application code on demand.
Responsible Consumption
Naturally, increasing access to a data environment increases the need to oversee that environment. DevOps’ principle of Responsible Consumption emphasizes the need for governance and standards in any environment where multiple users have access. On the current project, Thorogood used tools such as CloudWatch and SNS-based notifications that would proactively inform applications owners of the resilience and health of their apps. This added another level of controls and monitoring to the robust framework that the client had already developed.
Conclusion
The BI migration in question is a good example of how a DevOps framework can help a company move to a more Agile future. A DevOps approach creates a level of consistency and repeatability across environments that allows for development processes to run in parallel and across different teams. The result is a level of responsiveness that minimizes the consumption of both time and resources and allows a business to more quickly respond to its ever-changing needs.
