Dotfiles repository with submodule

I recently published my dotfiles on GitHub, and I wanted to talk about how I decided to manage them.

What’s a “dotfile”?

In short, we call dotfiles all the configuration files that you can find mostly under the home directory of Unix-like systems, such as Linux, macOS, and WSL under Windows. They are called dotfiles because usually (not always) their name starts with a dot.

Let’s make some examples:

• .zshrc / .bashrc - This is the configuration of your terminal shell if you use ZSH or Bash
• .gitconfig - This is your own user configuration that sets your name, email, and other things while working with Git
• .ssh/config - This config file contains settings about your SSH hosts, like authentication method, port, hostname, username, and so on

Why make a repo?

Dotfiles are something that one develops little by little during his life, fixing issues, and enhancing work experience. There are no jolly configurations that are good for everyone, dotfiles are personal.

Imagine losing them all when your laptop suddenly dies.

Less tragical, but more practical, let's say you have multiple PCs and you want the same configuration across all of them, that’s why a repo comes in handy.

You make a change on one of your files, you push that change, and you pull that on other PCs. Did something break? You go back to the git history and search for what caused that issue.

Now that we know the "what" and the "why", let's tackle the "how".

Creating a Dotfiles Repo

First of all, you should have a GitHub account, if not, create it now.
I suggest setting up SSH-based authentication on GitHub, so you don’t have to type your password every time you use a git command.

Note that I still use "master" as branch name, and not "main". If you use main just switch the name in the commands below.

This guide aims to create a public dotfiles repository, and a second private dotfiles repository to store sensitive data (such as SSH configurations).
The second repo will live inside the first repo as a git submodule, this way we can manage everything from one single folder.

On Github:

• Create a new repository, called "dotfiles", set it as public.
• Create a second repository, called "dotfiles-private", set it as private.

Now, on your PC, open the terminal, head to the home directory ( cd $HOME ) and clone your "dotfiles" repository as ".dotfiles":

1git clone git@github.com:<username>/dotfiles.git .dotfiles

Enter the cloned repo ( cd .dotfiles ) and add the private repo as a submodule that we will call "private":

1git submodule add git@github.com:<username>/dotfiles-private.git private

You should see a new .gitmodules file and private/ folder.

Enter in the submodule folder ( cd private ) and check out its master branch:

1git checkout master

Go back to the main directory of your dotfiles with cd ..

Set the submodule to always track the master branch when updating:

1git config -f .gitmodules submodule.private.branch main

Set the submodule update procedure to always merge the master branch, instead of creating a detached head

1git config -f .gitmodules submodule.private.update merge

Push everything to GitHub with git commit -am'Initial commit' and git push

Making changes

Let’s start by adding something, for example, our ~/.zshrc file.
Copy your ~/.zshrc file inside the ~/.dotfiles folder:

1cp ~/.zshrc ~/.dotfiles

Add and commit that file:

1git add . && git commit -m'Add .zshrc'

Now, to push, we need to use a particular git command:

1git push --recurse-submodules=on-demand

What is this?

When you have a submodule inside your repo if you push something you have to make sure that your submodule has been pushed. The --recurse-submodules=on-demand pushes the submodule before the main repository.

If you make changes inside the private folder, the procedure is similar.
Inside the private folder:

1git add . && git commit -m'Edit file X'

But then you go outside the private folder, and commit and push from the main dotfiles repo since Git needs to track that the inner submodule has been changed:

1cd .. # We are now inside .dotfiles/
2git add .
3git commit -m'Private submodule updated'
4git push --recurse-submodules=on-demand

Cloning the repo

As you may have guessed, to clone a repo with submodules we use a different command.

This is what you would use to clone a fresh copy of your dotfiles repo:

1git clone --recurse-submodules --shallow-submodules git@github.com:<username>/dotfiles.git .dotfiles

The --recurse-submodules makes sure to clone all submodules included in this repo.

The --shallow-submodules clones the submodule at its most recent state, without the entire git history.

Pulling changes from the remote

Again, there is a similar command to pull changes from the online Git repo:

1git pull --recurse-submodules

This was the basic setup I used to store my dotfiles, maybe in the future I’ll talk a bit more about what I did inside my dotfiles, but that’s for another time.

Thanks for reading!

More posts

Django, NGINX & Gunicorn

April 18, 2020

CentOS Server Setup

April 16, 2020