zsh history: extend and persist

Zsh can do so much awesome stuff with your history. Yet by default it doesn't save to file and only stores 30 items.

A history mechanism for retrieving previously typed lines (most simply with the Up or Down arrow keys) is available; note that, unlike other shells, zsh will not save these lines when the shell exits unless you set appropriate variables, and the number of history lines retained by default is quite small (30 lines). See the description of the shell variables (referred to in the documentation as parameters) HISTFILE, HISTSIZE and SAVEHIST in zshparam(1).

Zsh can do so much if you ask it: save lots of lines, store them on exit, merge multiple concurrent histories on save, store and merge incrementally! A dizzying array of options.

Bottom Line Up Front: Having done the following research, I'm adding some new options to my .zshrc:

# .zshrc
## History
HISTFILE=$HOME/.zhistory       # enable history saving on shell exit
setopt APPEND_HISTORY          # append rather than overwrite history file.
HISTSIZE=1200                  # lines of history to maintain memory
SAVEHIST=1000                  # lines of history to maintain in history file.
setopt HIST_EXPIRE_DUPS_FIRST  # allow dups, but expire old ones when I hit HISTSIZE
setopt EXTENDED_HISTORY        # save timestamp and runtime information

Let's start digging! Checking the zshparam manpage we see:

HISTFILE
The file to save the history in when an interactive shell exits. If unset, the history is not saved.
HISTSIZE <S>
The maximum number of events stored in the internal history list. If you use the HIST_EXPIRE_DUPS_FIRST option, setting this value larger than the SAVEHIST size will give you the difference as a cushion for saving duplicated history events.
SAVEHIST
The maximum number of history events to save in the history file.

As we continue digging, we find some of the other exciting options by looking at the HISTORY section of the zshoptions manpage: APPEND_HISTORY, EXTENDED_HISTORY, SHARE_HISTORY, INC_APPEND_HISTORY, HIST_SAVE_NO_DUPS, HIST_SAVE_BY_COPY, HIST_REDUCE_BLANKS, .

Once you start merging histories (APPEND_HISTORY, INC_APPEND_HISTORY or SHARE_HISTORY), you'll want to limit the duplications ( HIST_EXPIRE_DUPS_FIRST, HIST_IGNORE_DUPS, HIST_IGNORE_ALL_DUPS, HIST_SAVE_NO_DUPS).

I found INC_APPEND_HISTORY to be too confusing, merging all my current shells into one shared history, live as I go. I want !! or up-arrow to repeat the last command in THIS VERY SHELL.

APPEND_HISTORY
If this is set, zsh sessions will append their history list to the
history file, rather than replace it. Thus, multiple parallel zsh ses‐
sions will all have the new entries from their history lists added to
the history file, in the order that they exit. The file will still be
periodically re-written to trim it when the number of lines grows 20%
beyond the value specified by $SAVEHIST (see also the
HIST_SAVE_BY_COPY option).
INC_APPEND_HISTORY
This options works like APPEND_HISTORY except that new history lines
are added to the $HISTFILE incrementally (as soon as they are
entered), rather than waiting until the shell exits. The file will
still be periodically re-written to trim it when the number of lines
grows 20% beyond the value specified by $SAVEHIST (see also the
HIST_SAVE_BY_COPY option).
EXTENDED_HISTORY
Save each command's beginning timestamp (in seconds since the epoch)
and the duration (in seconds) to the history file. The format of this
prefixed data is:

`: :;'.
SHARE_HISTORY
This option both imports new commands from the history file, and also
causes your typed commands to be appended to the history file (the
latter is like specifying INC_APPEND_HISTORY). The history lines are
also output with timestamps ala EXTENDED_HISTORY (which makes it eas‐
ier to find the spot where we left off reading the file after it gets
re-written).
HIST_EXPIRE_DUPS_FIRST
If the internal history needs to be trimmed to add the current command
line, setting this option will cause the oldest history event that has
a duplicate to be lost before losing a unique event from the list.
You should be sure to set the value of HISTSIZE to a larger number
than SAVEHIST in order to give you some room for the duplicated
events, otherwise this option will behave just like
HIST_IGNORE_ALL_DUPS once the history fills up with unique events.
HIST_IGNORE_ALL_DUPS
If a new command line being added to the history list duplicates an
older one, the older command is removed from the list (even if it is
not the previous event).
HIST_IGNORE_DUPS (-h)
Do not enter command lines into the history list if they are dupli‐
cates of the previous event.
HIST_SAVE_NO_DUPS
When writing out the history file, older commands that duplicate newer
ones are omitted.

Data-Driven Documents with d3.js and cubism.js

Cubism.js: "A D3 plugin for visualizing time-series data".

Cubism can pull stats from graphite to produce pretty dashboard monitors. It is built on top of d3.js : Data-Driven Documents.

We're slowing putting stats into graphite at work. It'd be great fun to find the time to play with something like this , even better for someone to build me something shiny. Any takers? :)

remove a file from the most recent git commit with 'git reset'

How to quickly and easily remove a file from your most recent git commit while maintaing the changes in the working directory. Use 'git reset HEAD^' followed by 'git commit --amend'.

Caveats: Don't modify your commits if they've been pushed upstream or shared (unless you know why and how and ... no, just don't).

git reset HEAD^ -- $file
git commit --amend

Notes:

1. This technique works nicely if you need to modify a previous commit by wrapping it inside a 'git rebase -i ...' and then 'edit'ing the commit you want.
2. My right-hand prompt shows the VCS system (git), branch and git status. zsh's vcs_info is AWESOME.
3. Yes, I do have a smiley face in my prompt. It goes from a green happy face to a red sad face if the last command failed.

Short Example:

% git add a b c
% git commit -m "adds a and c"
# realize I didn't mean to include b!
% git reset HEAD^ -- b
% git commit --amend

Full Example:

[vm53@vm53]% git status                                 :) (git)-[master] ~/src/bar
# On branch master
nothing to commit (working directory clean)
[vm53@vm53]% echo "a" > a; echo "b">b; echo "c">c       :) (git)-[master] ~/src/bar
[vm53@vm53]% git status                                 :) (git)-[master] ~/src/bar
# On branch master
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       a
#       b
#       c
nothing added to commit but untracked files present (use "git add" to track)
[vm53@vm53]% git add a b c                              :) (git)-[master] ~/src/bar
[vm53@vm53]% git commit                                 :) (git)-[master] ~/src/bar
### editor opens with the old git message, modify if desired
[master 18d73bd] adds a and c
 3 files changed, 3 insertions(+), 0 deletions(-)
 create mode 100644 a
 create mode 100644 b
 create mode 100644 c
[vm53@vm53]% git status                                 :) (git)-[master] ~/src/bar
# On branch master  
nothing to commit (working directory clean)
[vm53@vm53]% git reset HEAD^ -- b                       :) (git)-[master] ~/src/bar
[vm53@vm53]% git status                                 :) (git)-[master] ~/src/bar
# On branch master
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#       deleted:    b
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       b
[vm53@vm53]% git commit --amend -m "adds a and c"       :) (git)-[master] ~/src/bar
[master c15220b] adds a and c
 2 files changed, 2 insertions(+), 0 deletions(-)
 create mode 100644 a
 create mode 100644 c
[vm53@vm53]% git status                                 :) (git)-[master] ~/src/bar
# On branch master
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       b
nothing added to commit but untracked files present (use "git add" to track)
[vm53@vm53]%                                            :) (git)-[master] ~/src/bar

Email for the High D (DiSC model)

What does a High D want in her email?

Direct. No salutation, no contact info. Bottom line up front. Short: 4 paragraphs of 4 sentences is an upper bound, one to two sentences preferred. Actionable. No story, no background. NO ATTACHMENTS unless you are specifically sending them something they asked for. BEAM INFORMATION INTO YOUR HEAD.

A high S/high C would call the same email too short, underspecified and cold/angry.

You've got a huge corpus to train from, read your boss's emails and reply using her style. Improves communication (ie, the actual ability to get your thought into their head) and appreciation.

Email and the High D from the Career Tools podcast from the Manager Tools guys.

See also Email and the High I. Looking forward to "Email and the High C" and "Email and the High S," I hope they get recorded and shared.

More Storm Videos

Six videos covering an hour-and-a-half of Nathan Marz and Ted Dunning talking about Storm at SF DataMining.