The Challenge: Scaling Secure Development
As engineering teams and codebases grow, managing sensitive information like API keys, tokens, and config files becomes increasingly challenging. These secrets can occasionally slip through, and resolving such issues like rotating credentials, updating pipelines, and cleaning up version control history post-deployment can be complex and time-consuming.
To address this, we needed a solution that could prevent such instances from occurring in the first place—by identifying them early in the development cycle.
Proactively managing this risk helps maintain development velocity and safeguards internal systems.
Across the industry, there have been several high-profile incidents where hardcoded credentials led to significant security failures. This highlights the importance of putting safeguards in place to prevent secrets and API tokens from ever entering version control history in the first place. Proactive prevention isn’t just a best practice—it’s critical to maintaining trust and protecting sensitive systems.
Tracing the Root Cause
We built an internal system to automatically flag secrets on every Pull Request. As part of validating its effectiveness, we sampled a few business-critical repositories and assessed the status of identified secrets. This exercise helped us surface a handful of active credentials, reinforcing the importance of early detection and the value of automated checks in strengthening our security posture.
Our Approach: Secure-by-Design with Pre-Commit Hooks
It was clear that we had to develop a solution at the pre-commit level to minimize the Time to resolve in case a valid secret is detected. That’s where pre-commit hooks come in.
A pre-commit hook is a script that runs automatically before you make a commit in Git. It checks your code for secrets, formatting issues, or test failures to catch problems early. If there are violations in the security rules, it can stop the commit from going through. This helps maintain code quality and prevent mistakes before the code is saved in the repository.
To deploy our chosen suite of pre-commit hooks consistently, we explored various methods and settled on a centralized, script-based approach. We created a bash script that fully automates the installation and configuration of the pre-commit hooks. By deploying this script through our device management system, we achieved our key goals: simplifying the setup process for developers by eliminating manual configuration, and ensuring a consistent environment across all systems.
Finding the Right Fit: Why We Chose TruffleHog
We tried with some pre-commit solutions like Talisman and detect-secrets. They were working well in theory but there were problems at scale like high false positive rate, less number of detectors/signatures, creating configuration files, etc. We found these issues when we installed the hook on a sampling basis on teams. It became evident that we needed to have better control and less configuration issues for a pre-commit hook. That’s when we tried Trufflehog!
TruffleHog struck the right balance between being strict enough to catch live secrets, giving minimal (<1%) false positives and being less configuration heavy on developers. We configured Trufflehog to catch only verified and known patterns of secrets. We tried it on a few internal repos, and automated the install/uninstall process to make rollout smooth. With this setup, the installation process required no manual intervention from developers—the script became active immediately after deployment and began flagging any detected secrets.
Deployment Strategy: Scaling Secure Commits Across Teams
To get the pre-commit hook running throughout the org, we developed a bash script that will install the pre-commit hook on all the repositories in a user's system by finding out .git files.
In each repo, it will create/update the .pre-commit-config file to have the Trufflehog pre-commit hook and install it.
For running the Trufflehog hook, packages like trufflehog and pre-commit are needed. These can be installed through Python but there might be several inconsistencies in Python versions among developers which can make the installation process faulty for some developers.
After some evaluation we decided to go with Homebrew (Open-source package manager primarily for macOS), as it is natively installable on MacOS and supports the dependencies required here without much hassle.
Problem: How can we ensure that the pre-commit hook is automatically set up for any new repositories cloned or created on a developer’s machine after the initial installation?
Git natively supports a post-checkout hook, which we can use here. The post-checkout Git hook is triggered after a successful git checkout command. We can configure a global post-checkout hook in `/Users/username/.git-templates/hooks` which will copy the script into a per repo basis when the post checkout hook is triggered.
While cloning a repo, Git checks out the default branch of the cloned repo and that would run the post-checkout hook here and initiate the script to create the pre-commit-config.yaml file for that repo. In cases where git init is run locally, the post checkout hook is triggered when a new branch is created from the base branch. So both the cases will be handled here.
We executed the rollout in phases, starting team by team, to observe any potential issues. Once we were confident there were no disruptions, we extended the deployment across the org using our device management platform.
V2: From Detection to Actionable Insights
The V2 of this project consisted of one of its kind dashboarding functionality for the pre-commit hook. For this we created a dashboard and exposed a webhook, where we will get the events when the pre-commit hook detects a valid secret. The webhook API is called from the developer’s instance when there is/are any valid secrets detected in the pre-commit. This is done in real time which helps to gain real time visibility and find out if it has been pushed into Git by bypassing the hook.
The security team can triage the findings received from the webhook and check if the secret has been successfully remediated. Once an alert is checked out by a team member, they can mark it as viewed to triage the finding. Additionally, they can mark the alert as Acknowledged/Bypassed based on the validity and status of the secret, whether it has been pushed into Git.
To effectively measure the influx of secrets, the dashboard offers an analytics section showing how many secrets have been identified by the pre-commit hook, and how many have been triaged/acknowledged/bypassed in the last n days/weeks. This view helps to view the general trend among developers and track effective metrics around the same.
What Changed: Security, Velocity, and Developer Experience
- Secret Prevention: Shifted secret detection to the "left," preventing secrets from ever being committed to the codebase history.
- Lower Remediation Overhead: Eliminates the time-consuming process of rotating committed secrets and lower risk of credential leakage.
- Improved Developer Workflow: Integrated security seamlessly into the commit process, providing immediate feedback to developers.
- Improved Visibility: The dashboard provides a real-time, organization-wide view of attempted secret commits, highlighting trends, hotspots, and potential training needs.
- Security Awareness: Regularly interacting with the hook reinforces secure coding practices regarding secrets management among developers.
Impact in numbers - Since the introduction of the pre-commit hook, we have prevented multiple instances of live secrets being committed to the code. This includes secrets from third party integrations and other internal pattern formats.
Beyond V2: Strengthening Security as We Scale
To further enhance our secret detection workflow, we’re working on enhancing detection rules and addressing bypass risks like:
Adding custom detection rules to cater to Meesho specific secret patterns and setting up an internal verification server.
Restrict the write privileges for pre-commit-config files to Admin role only.
These efforts are part of our broader goal to shift security left and bake it into our day-to-day developer workflows without adding unnecessary friction. This is an ongoing journey, and we’re excited to keep improving the developer experience while raising the security bar across Meesho.
