A common DevOps pattern is for platform engineering teams to own and maintain enterprise tools and templates, while delivery teams consume them. These teams have a strong interest in delivery team enablement and adopting their platform. When new tools roll out, they need to prioritize enablement. and ensure that teams know how to onboard and integrate into their existing ecosystem and embed best practices into their way of working.
For large enterprises, new tools can mean disruption:
- Dozens of teams — new work added to backlog
- Hundreds of engineers — new learning required
- Thousands of pipelines — code changes to their repositories
The sheer volume, context switching, and learning results in a big investment. Platform teams are usually understaffed and lack the capacity to provide this detailed hands-on support for delivery teams, especially with fluctuating demand. They may come up with roadmaps, best practices, and documentation, but still face a challenge to help teams adopt the tools and integrate into their way of working. Dojo can be the perfect ally for platform engineering teams to enable delivery teams on these migration efforts. It can be an effective way to start, establish, and scale a repeatable pattern for tools migration and adoption.
New Platform Engineering Tools! Now What?
On a recent project, we supported a Platform Security Automation team as they made significant changes in their tooling. After months of research and PoCs, they decided to switch from Veracode to Snyk for security scanning (we’ll keep our opinions and comparisons for another blog). Their goal was a shift-left approach to security to support the organization's overall goal of continuous delivery. Instead of a slow, nightly scan separate from the team's CI pipeline, security scans would happen in real-time during local development and their build pipeline. The instant feedback would bring significant improvements to the organization's security posture.
Snyk represented a continuous delivery model that placed the responsibility on the team to manage vulnerabilities. Platform Security saw this as more than a tool migration, but also an improvement to how team’s should develop software by building security into their delivery process.
New tools are great. But how do we get delivery teams to use them effectively? This is the essence of enablement.
Swapping out tools is the easy part (though still difficult). Adopting new tools is the real challenge. How can the organizations provide support to upskill teams and engineers on how to use these new tools effectively and show marked improvement is the real challenge? How can organizations ensure that tool migrations are worth the cost and show improvement?
Migration Scope
Once Snyk was accepted, Platform Security target a 6-month migration effort, motivated largely by their end of life date driven by annual renewals. This effort involved three primary goals:
- Offboard Veracode — driven by the end of life date
- Snyk onboarding and integration (local scanning, pipeline for individual apps, monitoring and configuration) — driven by new security standards
- Improve security — reduce existing vulnerabilities with emphasis on criticals and highs
Looking across the entire organization, the scope of migration quickly added up:
- Over 200 apps that were on Veracode. Their governance required all prod apps to be scanned. All these apps needed to be added to Snyk before being removed from Veracode.
- More than 60 teams, representing more than 400 engineers that needed to learn how to use Snyk and own security within their team and delivery process.
- Unknown number of existing apps that were not covered under Veracode, some of which were on cloud and some on-prem.
Dojo helped create a two-pronged adoption plan for both existing apps and new apps. As we worked with teams to migrate existing apps from Veracode to Snyk, we also wanted to stop incurring additional technical debt and ensure new applications used Snyk instead of Veracode. We partnered with the Core engineering team, who owns starter templates, to get Snyk into new apps by default.
Starting with Greenfield
The Core engineering team calculated that about 1 new app was created each day from their starter templates. Although this may seem nominal, technical debt would quickly incur over the days and weeks as these new apps had Veracode instead of Snyk.
To get Snyk embedded into Core’s starter kits, this work included the following for each of Java, Kotlin, Node, ExpressJS templates:
- Removing 58 vulnerabilities based on Snyk scans. Presently there were vulnerabilities in the templates, resulting in broken builds for new teams.
- Basic logic to remove Veracode and add Snyk. Veracode and Snyk fundamentally scan differently. The supporting pipeline library needed changes as well as the consumer template pipeline.
- Upskilling the Core team on Snyk. The Core team was the first team to go through the Dojo, focusing on product improvements, security upskilling, and migrating from Veracode to Snyk. They needed to maintain their starter kits and pipelines as a product. This meant understanding and dealing with vulnerabilities from Snyk scans.
- Documentation. As teams would now receive Snyk out of the box, proper documentation and user guides would need to be added to Core’s Backstage.
Why Dojo?
Dojo is an immersive learning experience. Our DevOps experts improve teams at their pain points. Instead of external training separate from real work, Dojo embeds coaches and engineers with delivery teams to align challenges and improvements with their backlog.
Dojo is our proven vehicle for transformation, providing DevOps experts in coaching and engineering to enable new and modern ways of working.
Dojo is an effective vehicle in any transformation, from on-prem to cloud migration, to small batch delivery, to monolith to microservice, to tool adoption, and anything in between. This model was an obvious choice to support the tools migration with the following benefits:
- Developer experience. Dojo works with teams on their actual work day-to-day. This gives us valuable insight into the life of engineers. We’re able to anticipate the challenges and questions with using new tools and how it fits within their existing behaviors, norms, and delivery workflow. We use these inputs to define the default user experience should be.
- White-glove experience. Dojo is hands-on with teams, armed with relevant expertise, documentation, and troubleshooting. This keeps teams from spinning their wheels and waiting on support tickets.
- Pull together the right people. Dojo is staffed with DevOps experts and also keeps close ties with platform teams. We’re able to bring the right people into the right context to help solve problems.
- Feedback for shared services teams. Dojo provides quick and honest feedback to platform teams on the usability and functionality of their services. This promotes iterative development, enables improvement for future teams, and overall improvement to the organization.
We brought our engineering and enablement expertise to partner with Platform Security Automation and Core Engineering. This was a perfect opportunity to create a FlashBuild Dojo offering. These are shorter, targeted engagements focused on a single objective.
Playbook for Tools Migration with Dojo
Phase 1: Establish Partnership
Louis, the Tech Lead for Platform Security Automation, reached out to us based on Dojo past success stories. Through a series of consults and conversations, we decided this was an obvious candidate for an extended, dedicated engagement. We also ensured that this effort was aligned with organizational goals and priorities. Before we start any Dojo engagement, we facilitate a Charter to establish these goals and working norms.
Identify Dojo coach pair
We allocated two coaches for this effort. Ethan was an engineer who became a strongly identified SME, and I worked closely with Platform on strategy and planning. We used this opportunity to showcase our expertise as well as delve deeper into a specific challenge regarding security tooling and product rollout.
Add Tools to our internal apps
Together, the Dojo coach pair implemented Snyk tooling to see how teams would interact with it. This involved local development and IDE integration, pipeline setup and configuration, and setting defaults across the UI app. Additionally, as teams were migrating from Veracode, we spent a lot of time playing developer advocacy for how to use Snyk. We brought our opinionated view for developer experience and helped define this across the software delivery pipeline.
Align priorities across shared services teams
Platform Security automation is one of several platform teams. They were interested in security, rolling out Snyk, and retiring Veracode. However, as Dojo inherently holds developer empathy, we saw additional players in this effort. We pulled together the Core team, responsible for maintaining templates for new projects. Together, we defined and prioritized the work needed to get Snyk into these templates, from boilerplate code to security improvements to tool settings and defaults.
Develop a roadmap of engagements and milestones
Platform Security was driven by an end of life date with Veracode. Together, Dojo worked with Louis to look at timelines for an initial pilot, engagements for critical teams, and greenfield adoption through starter kits. We roadmapped further as we looked ahead with Snyk at scale. After incorporating feedback and improving documentation throughout the pilot, we anticipated self-service and workshops to accelerate the migration. This roadmap was helpful to begin conversations with teams and managers to get their teams lined up for Dojo engagements.
Establish key metrics
As with all of our engagements, Dojo needs to show its impact. This can be done through OKRs, improvement measurements, goals and outcomes, or sheer volume metrics. The valuable metrics we agreed on for the Snyk effort were: number of engineers upskilled, number of pipelines with Snyk scans, percentage of repos monitored by Snyk, number of vulnerabilities resolved, percentage of repos with no vulnerabilities (criticals and highs), and percent reduction in criticals and highs. Initially this data was captured manually and fed into an Excel sheet, but as we evolved our practice we developed a script to gather the data.
Phase 2: Pilot
Once we established our key goals, challenges, and work norms, the next piece was to pilot an engagement. We talked with low-risk teams and used this opportunity to create materials and define a scalable Dojo FlashBuild engagement. The goal of this phase is to prove out the Snyk implementation, test the engagement model, and then begin communications and marketing channels for the scaling phase.
Identified pilot teams
Selling an engagement can be a challenge, especially when it’s a pilot. To make it easier, we talked with friendly's — teams who have already gone through the Dojo and teams that work closely with the Platform Security team. Together we identified four teams with four 2-3 day engagements for our pilot. Each team went through the typical Dojo lifecycle including consult and charter. We inserted an additional pre-Dojo call for onboarding and checklist, ensuring the work and priorities for the FlashBuild.
Create materials
I used this time to create learning content for our Dojo portal, as well as engagement information and marketing decks for consults. Additionally, Ethan used this capacity to add interactive material on our Dojo Portal and Confluence, including tool onboarding, CLI setup, IDE configuration, and pipeline code snippets.
Execute Pilot with Platform Security
We typically staff our Dojos with 2 coaches. For these Snyk pilots, we brought in Louis and 1-2 additional Platform Security engineers. This is the power of Dojo, bringing together the right people to solve problems. Getting help in real-time to remove blockers, solve troubleshooting, or explain tool nuances is what makes Dojo impactful. In addition to helping support Dojo, the additional engineers received instant feedback and observed how we run immersive learning sessions. They would see how we facilitate team problem solving, prioritization, pairing and mobbing sessions that would equip them to run workshops in the future.
Build out Dojo offering
Each pilot engagement was a learning opportunity to take feedback, improve our materials, and sharpen our marketing. We used this time to create a standardized set of goals and outcomes for our charter, as well as improve our data gathering and tracking process. We streamlined our pre-engagement process including onboarding, Jira story templates, and work prioritization for the engagement. Finally, we added better content and visualizations on our Dojo portal for coaches to facilitate key topics in the Dojo.
Phase 3: Scale
As we finished the pilot phase, we gained confidence in Snyk’s implementation and usage, as well as the coach's ability to deliver content. We still ran into snags, but were able to resolve them quicker and give better explanations and solutions. Ultimately we wanted to remove Dojo as a bottleneck for Snyk onboarding and reserve capacity for high value and challenge engagements. Once at scale, the goals are to develop a sustainable, repeatable model with the widest impact. This means engagement styles may need to change and responsive support systems need to be developed.
Bi-Weekly Touchpoints
Dojo continued executing engagements based on the roadmap. To keep up to date on engagement changes, product feedback, and new milestones, we set up a bi-weekly touchpoint. This helped keep our planning in check as well as
Strategic engagements
Once the Snyk engagement model was proven and materials were sharpened, we devoted our energy to high value and high impact teams and applications. We decided to initially work with a single domain or line of business. As we worked with the same engineering managers and tech leads, our partnership and trust grew, paving the way for a better and more impactful engagement.
Leadership Dojos
We facilitated several 1 hour Leadership Dojos with domain leads, including Director, Engineering Manager, Tech Lead. We used this time to communicate changes with Snyk, technical debt that will be incurred, and how Dojo can support them. This helped bring all the communication into one conversation, representing several teams and common questions like planning capacity for Dojo, new work introduced, and what we need from them.
Workshops
Dojo helped create a model for Platform Security Automation to facilitate aided onboarding through 2-hour workshops with tech leads. This was a compromise of quality and scale. To get more teams onboarded to Snyk, engineers ran a train-the-trainer workshop to ensure teams were onboarded, worked through a single vulnerability and pipeline change, and then knew where to go for information. Workshops are a great option as a Dojo practice and tooling migration is at scale, however we caution with going to workshops too soon as we believe it is much less effective at impactful results.
Self-service
Once we worked through initial common problems, we opened Snyk to self-service. This allowed mass onboarding with minimal issues. Engineers were also more available to support these issues as they were no longer heavily involved in Dojo.
Slack forum
We opened a public Slack forum that was closely monitored by Platform Security Automation and Dojo coaches. We had team members join this channel on Day 1 of the Dojo so that they were immediately introduced to the community and support system. As others onboarded through self-service or workshops, the channel continued to grow and become a vibrant knowledge base and communication hub.
Did Migration Work?
Four months into the tools migration, we’ve received overwhelmingly positive feedback from stakeholders, manager, and developers. Louis has been the Dojo’s biggest supporter.
The Dojo has been an incredible partner supporting the effort to not only migrate tools to Snyk but also teach folks how to use these tools to improve the security of our systems.
— Tech Lead for Platform Security Automation
Really glad that we did the Dojo for the Snyk work instead of self-service. This would have taken at least twice as long if we did it ourselves, and now everyone on the team knows how it works.
— Tech Lead for Delivery Team
Beyond anecdotes, we were proud of the following accomplishments:
- 15 teams with 117 engineers hands-on experience with Snyk and security best practices
- 41 pipelines with Snyk scans
- 78% reduction of criticals and highs in Dojo team’s apps
- 453 vulnerabilities total resolved, including 11 criticals and 154 highs
We initially allocated a Dojo track for two months, including one engineer and one product coach. The feedback from our client — both the product owner and our stakeholder — was overwhelmingly positive. After seeing the success of our initial pilot, they asked for more effort into the Snyk migration.
Repeatable model
More engagements. After experiencing a Dojo engagement themselves, one of the platform engineering teams wanted to have a similar model as they are migrating Kafka tooling to Confluent. Similarly, the pipeline team observed the success and metrics from our Snyk engagements and wanted to follow a pattern for their migrations to using GitHub and GitHub Actions.
Rotating coaches. Once we ran through the pilot phase, we rotated other Dojo coaches and engineers through these FlashBuilds as well as embedded the content into our other Dojo engagements. This helped reduce any bottlenecks and spread the knowledge across, while reducing coaching fatigue on similar engagements.
Robust feedback. Although Snyk is enterprise-ready, there were a lot of nuances with the tool. Dojo was crucial in providing hands-on guidance across multiple project types, Snyk quirks, and customer-specific implementation. This feedback would not have been surfaced and actioned without the white-glove experience that Dojo coaches provide.
Platform engineering tools migrations are difficult, but organizations must continually evolve in order to compete in the industry. Modernizing an engineering organization often requires new tooling to continually improve security, speed, and quality. It takes months of research, spikes, vendor conversations, negotiations to make this decision. And then starts the work of adoption and migration, along with best practices and improvement. Dojo can play a vital role in bringing together platform teams and delivery teams to modernize their engineering tooling and practices.
Liatrio strongly believes that Dojo is the most effective way to implement transformation. Learn more about Dojo, how it works, and read our case studies.
