Removing Sensitive Data From Git History

June 21st, 2020

I would like to think that we have all be there: accidentally pushing a secret (access token, password, connection string, etc) to your remote git server and immediately proceed to start to freak out. Or maybe its just me 🤷🏽. Anyhow, luckily for you I have indeed done it before and found a great tool to re-write history! BFG is a tool that allows you to scrub your git repositories clean and remove all sensitive data that you might have accidentally committed and luckily is extremely simple to use and extremely fast 🚀. Go ahead and download the jar file & make sure that you have java installed on your machine in order to run it.

Removing Sensitive Data (Data has feelings too)

Remove the secrets from your files & commit into your master branch. The reason we have to do this is because the HEAD commit is considered a protected commit and the BFG tool will not touch it.
```
git commit -m "Oops"
```
Get a fresh clone of your repository. You will want to use the --mirror flag here since we are only want the git history & not all of the actual files in your repository.
```
git clone --mirror your_repo_url
```
Save a copy of this new clone just in case anything bad happens.
Create a new .txt file and add the sensitive information that would want to remove. Each entry should be on a separate line. BFG will search throughout your entire git history for any occurrences of these keys.
```
superawesomeapikey1
password123
```
Run the command to replace the all occurrences of you secrets with ***REMOVED*** .
```
java -jar bfg.jar --replace-text secrets.txt  your_repo.git
```
— Note: There are many other commands that you can run instead of replacing the text. Check out their documentation for all of the commands.
Change your directory to your git repo and run the following command to strip out all of the "dirty" data.
```
cd your_repo
git reflog expire --expire=now --all && git gc --prune=now --aggressive
```
This command is actually two commands concatenated together by the && operator. This appears to be a bit more complicated but lets try to break it down:
```
git reflog expire --expire=now --all
```
First, reflog here is short for "Reference Logs". Reference logs just keep track of when you git history (branches, commits) where last updated in your git repository. expire --expire=now in this command is tell git to prune all older git reflog entries. Finally we are adding the tag --all to tell git that we want this operation ran on all references.
```
git gc --prune=now --aggressive
```
Secondly, we run the git gc command. This command is essentially cleaning up your repository. --prune=now tells git to remove all references to any orphaned or unreachable git objects. --aggresssive is telling git to sacrifice speed in order to clean everything as well as it can.
Once everything looks good in your local repository it is time to push it back up to your remote repository with a simple:

git push

Now all of the secrets that you specified in the secrets.txt file will be replaced with ***REMOVED***

and you will have a squeaky clean git history! 🎉 One important thing to note here is if you are working on a team with other people you are going to want to make sure that they do a new git clone . Thanks for reading! And make sure to give BFG some love and give them a star on their repo on GitHub!

Follow me on twitter @jeff_codes