Configuring Bash to Make Your Command History Useful

Oct 12, 2024

Bash has been around for a long time. Newer shells (like Fish) come with default settings that make sense in 2024, but Bash is stuck with defaults that were first decided on decades ago.

Julia Evans wrote about this recently, pointing out in particular how the command history in Fish works much better by default than it does in Bash.

I’ve never been happy with how the command history works in Bash. I often find myself searching backward for commands I know I ran recently only to come up empty handed. Or sometimes I run a command in one window and wonder why I can’t recall it from my command history in another window.

I’ve been able to improve things by changing some of Bash’s default settings and learning how the command history feature works. I had to spend some time reading through the Bash man page to figure it all out, so I’m sympathetic to anyone who’d rather use a shell that just works. For those wanting to stick with Bash: You can make it better, but not without some patience for Bash’s quirks.

The In-Memory History

The two most important settings governing how the command history behaves are the HISTSIZE and HISTFILESIZE shell variables. There is some debate about how these two settings interact and whether it ever makes sense to set them to different values. To understand what these two settings do, it’s helpful to know how Bash saves the commands you run.

Bash saves every command you execute to a command history that lives in memory. If you imagine a simple world in which you only ever have one instance of Bash active (say, you only ever use a single window or tab in your terminal emulator), and also that you use that instance of Bash forever (by never shutting down your computer), then this in-memory command history would serve all your needs. Every command you search for could be found in that in-memory buffer as long as you actually ran it.

Memory isn’t infinite, of course, so it makes sense to have an upper limit on how many commands are stored in this buffer. That’s what the HISTSIZE setting controls. Once the in-memory buffer contains HISTSIZE commands, each new command saved to the history pushes out the oldest command. In this simple world of a single, immortal instance of Bash, the HISTSIZE setting is all we need.

By default, HISTSIZE is set to only 500, which likely isn’t sufficient to retrieve a command you ran a few days ago or maybe even just yesterday. Modern computers have enough memory to spare that you can bump this up to something like 100,000 without issue. Assuming you run 250 or so commands a day, that should be enough to hold more than a year’s worth of shell commands.

To make Bash’s command history more useful, bumping up HISTSIZE to 100,000 or more is the first thing you should do.

The History File

In reality, people stop and start instances of Bash all the time and might have several running at once. Let’s ignore the possibility that multiple instances of Bash could be running at once for now and just consider what happens when you exit and restart Bash.

If Bash only stored commands in memory, this would lead to a situation where each instance of Bash would have its own entirely separate command history. Each time you opened your terminal, you’d start with a completely clean slate. And every time you exited your terminal, your command history for that session would be lost.

Bash prevents this by storing your command history in an additional location: on disk. There is a Bash setting called HISTFILE that governs where on disk your command history is stored. The default location, if you haven’t defined this variable, is ~/.bash_history.

When you exit an instance of Bash, it saves all the commands from the in-memory command history to this file. Next time you start an instance of Bash, the in-memory command history is initialized with the contents of the file. This preserves your command history between Bash sessions.

Disk space, like memory, isn’t infinite, though certainly we have much more of it available. So the HISTFILESIZE setting is the on-disk analog of the HISTSIZE setting—it defines an upper limit on the number of commands to store on disk. The Bash man page says that the history file is truncated to contain no more than HISTFILESIZE lines when 1) the file is read into the in-memory command history of a new Bash instance and 2) the file is written to after a Bash instance exits. So the file should never contain more lines than the specified upper limit.

Since we have so much more disk space available than memory, it might seem natural to set HISTFILESIZE to a higher number than HISTSIZE. You have to remember though that when you search backward through your command history with Bash, you are always searching backward through the in-memory command history of the current Bash process. The in-memory command history never contains more than HISTSIZE commands, even if those commands were read from the history file when the Bash process started. So setting HISTFILESIZE alone to a really big number does not mean that you now have a long command history. You would have to ensure that HISTSIZE is set to a large number as well, otherwise all those extra commands in your history file just won’t fit in the in-memory buffer.

I’m sure this confuses many people. I’ve had this misconfigured for years.

There are never more than HISTSIZE commands available in the Bash command history. If you set HISTSIZE to 1000 and HISTFILESIZE to 2000, you will have up to 1000 commands in your command history, not 2000.

So why is there a separate HISTFILESIZE setting at all? Some people might want to set HISTFILESIZE to a lower number than HISTSIZE if they are concerned about disk space. Alternatively, you might want HISTFILESIZE to be larger than HISTSIZE if you plan to search the history file using an external tool. I think there’s at least one good use case for this that helps you manage command histories from multiple active shell sessions. Let’s talk about that now.

A Global Command History

If you’re running more than one shell session at a time, even if you’ve set HISTSIZE and HISTFILESIZE to large numbers, you can still end up in a situation where you know you ran a command recently but aren’t able to find it in your command history. This will happen whenever you run a command in one shell session then try to search for it in the other.

The problem is that, for a command executed in session A to appear in the command history for session B, session A first has to write the command from its in-memory history to the history file and then session B has to read it. Usually these writes and reads only happen when sessions exit and start, so two active Bash processes will never merge their command histories, at least not while they are running.

There are ways to force Bash to write and read the command history file after every single command. This ensures the in-memory command history of every session and the on-disk command history are always in sync. It’s not obvious to me though that this is the best solution. Sometimes you have a shell session open for some particular category of activity and actually it’s useful for the commands you run in that session not to show up in the command history of your other sessions. If you are constantly writing and reading the command history file, you only have a single, global command history.

Here’s the setup that I think makes more sense. Treat the in-memory command history of each Bash session, governed by HISTSIZE, as your “local” command history. Then set HISTFILESIZE to a number many multiples of HISTSIZE. The history file will be your “global” history, which you search when you are looking for a command that you ran a long time ago or in another session.

You should also set the histappend shell option, which ensures that Bash always appends to the history file and never overwrites it in its entirety. You can do this by adding shopt -s histappend to your .bashrc.

Because HISTSIZE is smaller than HISTFILESIZE, you can’t use Bash to search backward through your global command history. Instead, you only use Bash to search through your local command history. When you want to search through your global command history, you use an external tool (like fzf) to search the history file directly. I have a nice shell alias set up to do that for me.

The nice thing about this setup is that you get two distinct search modes handled by two distinct commands. When you want to find a command you know you ran in your current session, you can do that. If you want to expand the search to your global command history, you can choose to do that too.

There’s still one piece missing here though—we have to ensure that all our commands from our local command histories are inserted into the global command history at some point. Remember that, by default, this won’t happen on its own until those Bash sessions exit.

One way to do this is to run history -a. The history command is a Bash builtin that helps you manage the in-memory command history. Running history -a appends the current in-memory history to the history file. If you start looking for something in your global command history and realize it’s not in there because the command is still in the local history of another shell session, you can go to that session, run history -a, then resume your global search.

I think this approach involves the least magic. An alternative would be to take the approach I alluded to above of forcing Bash to write and read the command history after each command it executes—except I think you should only configure it to write and not to read. That ensures that you global command history is always up-to-date without polluting your local command history with commands that you don’t want to see. You can do this by setting PROMPT_COMMAND=history -a; whatever you have set as PROMPT_COMMAND will be run each time Bash displays its prompt to you.

Phew! After all that, you get a command history that sort of works. Just not quite as well, of course, as things work out of the box in Fish. I’ve been keen to stick with Bash because of its ubiquity, but truthfully the more I have to configure it to get the experience I want the more I think I should just make the jump to something new.