It is not uncommon for projects or products to rely on external services to achieve a task or solve a problem. When an external service is being used, you will likely need some sort of token or key to authenticate with that service. During local development, it may be fine to store these tokens or keys in your local machine for a faster feedback loop. However, if these secrets manage to end up in your repository, especially if it's public, then anyone with read access to the repository will be able to view these secrets and access its corresponding service along with any privileges associated with them. Secret scanning is a GitHub Advanced Security (GHAS) feature that aims to be a developer-first solution for identifying secrets that have made their way into your repositories.
How Secret Scanning works
When secret scanning is enabled for a repository, the repository’s entire git history on all branches are scanned for secrets. GitHub has a “Secret scanning partner program” where service providers can partner with GitHub and provide them with their secret token formats, which will then be scanned for during the secret scanning process.
The above diagram outlines the secret scanning lifecycle. Any changes in the codebase that get pushed up to a public Github repository will automatically be scanned for partner patterns. If GitHub detects a match, the results are sent to the corresponding service providers for verification that a match was indeed found. Once they confirm the match, they have a couple options; they can either: revoke the secret, issue a new secret, or contact you directly. For organizations with GHAS enabled, this process is the same with the added benefit of being alerted in the Secret scanning alerts section in the Security tab of your repository.
Push Protection
While detecting secrets after a push is valuable, we can actually empower developers to maintain security in their repositories even more and shift the security process further up in the delivery lifecycle by enabling push protection. This feature allows developers to make remediation choices by blocking any pushes that contain secrets that are supported by service providers or custom patterns. When such a push is detected, a developer may choose to remove the secret from the commit, or head over to a user-friendly interface to allow the push.
Custom Patterns
So what if your organization also wants to keep track of custom secret formats that are used internally? Secret scanning still has you covered! As long as you have a GHAS license and secret scanning enabled, you can also define custom patterns, which are specified using regular expressions that secret scanning will look for in addition to the default patterns.
Reviewing and Remediating Results
As you can imagine, the results you get back from secret scanning contain highly sensitive information that only relevant people or teams should have access to view. Pre-defined roles can be assigned to members and teams of an organization to enforce this security boundary. While there are only 5 pre-defined roles (along with any custom defined roles) it can get tedious assigning security team members to the same roles across multiple repositories. For this reason, GitHub created a security manager role that is assigned at the organization level. Team members assigned to this role not only have read access to all repositories, but also permissions to access and manage security alerts across the organization.
As we have seen briefly in All GHAS, No Brakes!, the Security Overview is a convenient hub for security teams and admins to see which GHAS features are enabled and analyze their results across the organization and its repositories.
From here, we can identify the amount of repositories with secrets detected and the amount of secrets that have been scanned. We can also sort the repositories by Secret scanning alerts, allowing us to easily access any repositories that contain secrets and view their individual secret scanning alerts for remediation.
Conclusion
When fully utilized, secret scanning is a powerful developer-first tool that implements security in every step of the delivery process. For larger organizations that are looking to implement GHAS, be warned that implementing secret scanning can be daunting at first. The best approach is to educate teams on what is available to them and set standards for remediating results. Taking your time is key.
Remember that this is only one feature out of three that GHAS has to offer. Follow along with the rest of the series to find out more about how to secure your codebase!
