zsh history: extend and persist

Zsh can do so much awesome stuff with your history. Yet by default it doesn't save to file and only stores 30 items.

A history mechanism for retrieving previously typed lines (most simply with the Up or Down arrow keys) is available; note that, unlike other shells, zsh will not save these lines when the shell exits unless you set appropriate variables, and the number of history lines retained by default is quite small (30 lines). See the description of the shell variables (referred to in the documentation as parameters) HISTFILE, HISTSIZE and SAVEHIST in zshparam(1).

Zsh can do so much if you ask it: save lots of lines, store them on exit, merge multiple concurrent histories on save, store and merge incrementally! A dizzying array of options.

Bottom Line Up Front: Having done the following research, I'm adding some new options to my .zshrc:

# .zshrc
## History
HISTFILE=$HOME/.zhistory       # enable history saving on shell exit
setopt APPEND_HISTORY          # append rather than overwrite history file.
HISTSIZE=1200                  # lines of history to maintain memory
SAVEHIST=1000                  # lines of history to maintain in history file.
setopt HIST_EXPIRE_DUPS_FIRST  # allow dups, but expire old ones when I hit HISTSIZE
setopt EXTENDED_HISTORY        # save timestamp and runtime information

Let's start digging! Checking the zshparam manpage we see:

HISTFILE
The file to save the history in when an interactive shell exits. If unset, the history is not saved.
HISTSIZE <S>
The maximum number of events stored in the internal history list. If you use the HIST_EXPIRE_DUPS_FIRST option, setting this value larger than the SAVEHIST size will give you the difference as a cushion for saving duplicated history events.
SAVEHIST
The maximum number of history events to save in the history file.

As we continue digging, we find some of the other exciting options by looking at the HISTORY section of the zshoptions manpage: APPEND_HISTORY, EXTENDED_HISTORY, SHARE_HISTORY, INC_APPEND_HISTORY, HIST_SAVE_NO_DUPS, HIST_SAVE_BY_COPY, HIST_REDUCE_BLANKS, .

Once you start merging histories (APPEND_HISTORY, INC_APPEND_HISTORY or SHARE_HISTORY), you'll want to limit the duplications ( HIST_EXPIRE_DUPS_FIRST, HIST_IGNORE_DUPS, HIST_IGNORE_ALL_DUPS, HIST_SAVE_NO_DUPS).

I found INC_APPEND_HISTORY to be too confusing, merging all my current shells into one shared history, live as I go. I want !! or up-arrow to repeat the last command in THIS VERY SHELL.

APPEND_HISTORY
If this is set, zsh sessions will append their history list to the
history file, rather than replace it. Thus, multiple parallel zsh ses‐
sions will all have the new entries from their history lists added to
the history file, in the order that they exit. The file will still be
periodically re-written to trim it when the number of lines grows 20%
beyond the value specified by $SAVEHIST (see also the
HIST_SAVE_BY_COPY option).
INC_APPEND_HISTORY
This options works like APPEND_HISTORY except that new history lines
are added to the $HISTFILE incrementally (as soon as they are
entered), rather than waiting until the shell exits. The file will
still be periodically re-written to trim it when the number of lines
grows 20% beyond the value specified by $SAVEHIST (see also the
HIST_SAVE_BY_COPY option).
EXTENDED_HISTORY
Save each command's beginning timestamp (in seconds since the epoch)
and the duration (in seconds) to the history file. The format of this
prefixed data is:

`: :;'.
SHARE_HISTORY
This option both imports new commands from the history file, and also
causes your typed commands to be appended to the history file (the
latter is like specifying INC_APPEND_HISTORY). The history lines are
also output with timestamps ala EXTENDED_HISTORY (which makes it eas‐
ier to find the spot where we left off reading the file after it gets
re-written).
HIST_EXPIRE_DUPS_FIRST
If the internal history needs to be trimmed to add the current command
line, setting this option will cause the oldest history event that has
a duplicate to be lost before losing a unique event from the list.
You should be sure to set the value of HISTSIZE to a larger number
than SAVEHIST in order to give you some room for the duplicated
events, otherwise this option will behave just like
HIST_IGNORE_ALL_DUPS once the history fills up with unique events.
HIST_IGNORE_ALL_DUPS
If a new command line being added to the history list duplicates an
older one, the older command is removed from the list (even if it is
not the previous event).
HIST_IGNORE_DUPS (-h)
Do not enter command lines into the history list if they are dupli‐
cates of the previous event.
HIST_SAVE_NO_DUPS
When writing out the history file, older commands that duplicate newer
ones are omitted.