There are as many blogs about software quality as there are ways to order a drink at a bar (I’ll take -2147483648 beers, please). This one is about how we here at Automox decided to evolve our approach to quality as Automox grew from sassy startup to industry leader.
The Roots
To my pleasure as a lifelong QE guy, Automox has always put a priority on Quality. We Invested early in Quality Engineers (QE), Software Development Engineers in Test (SDET) and an automation framework (note: these two roles are often differentiated to indicate manual vs automation-focused tester, for the remainder of this article I will just refer to our group collectively as QE). When I started two and half years ago, I was the third QE in an engineering team of less than twenty. We were able to build a fairly mature CI/CD pipeline with test automation gates and push-button deploys over the following year. But as the engineering organization grew to over eighty employees, we needed to change how we approached testing, release, and quality as a whole. We identified three pillars of this transformation:
- A shift to a Quality Assistance model.
- The formation of a QE Operations team.
- The empowerment of each development team to test and release their code independently.
The Evolution
Part 1: Quality Transformation Assistance
As Automox began to grow, we embedded QEs within the feature teams to be the gatekeepers of release, the owners of quality. And if I may so myself, we have a kick-butt team of Quality Engineers (by the way, we’re still hiring, come join us!). They provided invaluable product scrutiny, but also created a dependency on the QE as an individual to approve changes. The need for development teams to own quality is not a new concept. Many organizations have largely gone the way of removing the distinction between QE and product developers, and we knew we needed to increase engagement of all engineers in quality processes. We also recognized, however, that quality engineering is a skillset, and having people dedicated to it only improves overall product quality and customer confidence in Automox. So we went the hybrid route, following in the path of Quality Assistance.
What this means in theory is that everyone on a feature team is involved is testing. What this means in practice is that each team has an embedded QE, and that individual is engaging the entire team to be involved.
Everyone is empowered to participate in test planning, test validation, and writing automation. Adopting this model is a cultural change, which means it will take time for everyone to be comfortable in their role. We want the QE to keep doing what they’re good at and feature developers to keep doing what they’re good at, but we want everyone to be involved. In the near-term, the embedded QE will help train up the feature developers on what resources exist in the automation framework to leverage, and review test plans and test PRs. These dependencies ease up in the long-term as people become comfortable with their roles and expectations, but the QE is always there to assist.
Part 2: QE Operations
The second pillar of our evolution arose out of the need to support all the other quality-related tasks, which fell outside of feature team work. Initially, all of our QEs contributed as necessary to the automation framework. The load and performance testing work was my side hustle. We learned what we needed to in order to create our own infrastructure, and punted what we couldn’t figure out to our amazing SRE team (who were equally busy). In addition, we had several Quality Engineers whose day-to-day tasks included release duties. These engineers have deep knowledge of the existing regression test suites, as well as deployment process and monitoring. We decided to give all these tasks a home, and formalize a team around them. Combining these forces created a standalone, multi-faceted, highly-functional team we dubbed the QE Operations Team, or QEOps. Tasked with maintaining and improving the framework, auditing and correcting regression tests, and a variety of other quality enablement tasks allowed them all to work with more focus, while also allowing the embedded QE to focus on their feature team’s work.
Part 3: Breaking the Monolith
Let me start by saying, there was way more monolith talk in the past year than I expected. But I’m going to stick to talking about large releases and not mysterious obelisks. This last pillar is not strictly an effort led by the Quality team, but it is a large part of the quality evolution, and it is enabled by the first 2 pillars. As mentioned previously, our existing release process involved release captains from the Quality team. We released all our services, once per day, following a release sync where the release manifest was reviewed and test results were analyzed. By implementing Quality Assistance and the QEOps team, we are able to break these barriers and enable feature teams to control their own deployments. The Quality Assistance piece allows us to efficiently implement automation for all features, while the QEOps team will ensure continuous testing is stable and dependable. Together, we achieve the velocity and test confidence required to implement microservice deployment, enabling each team to deploy when they need, as often as they need.
Conclusion
At Automox, we are committed to increasing our customer’s security confidence, and to do that we need to have total certainty in what we produce every day. As our business changes and grows, we must adapt with it, by enabling all our teams to efficiently deliver the highest quality code. We are growing rapidly, with no plans to slow down, and I believe we have positioned ourselves to do so the right way. By the way, I want to change my order: I’d like 🔥🔥 beers, please (got to check those unicode orders).
About Automox Automated Patch Management
Facing growing threats and a rapidly expanding attack surface, understaffed and alert-fatigued organizations need more efficient ways to eliminate their exposure to vulnerabilities. Automox is a modern cyber hygiene platform that closes the aperture of attack by more than 80% with just half the effort of traditional solutions.
Cloud-native and globally available, Automox enforces OS & third-party patch management, security configurations, and custom scripting across Windows, macOS, and Linux from a single intuitive console. IT and SecOps can quickly gain control and share visibility of on-prem, remote and virtual endpoints without the need to deploy costly infrastructure.
Experience modern, cloud-native patch management today with a 15-day free trial of Automox and start recapturing more than half the time you're currently spending on managing your attack surface. Automox dramatically reduces corporate risk while raising operational efficiency to deliver best-in-class security outcomes, faster and with fewer resources.
