Customizing Zsh (Part 1): Hooks and RPrompt

This post follows my post on the zsh macros, and explain how to use the zsh hooks "preexec", and how to customize your (right) prompt to give information about something that changes (current branch of git, date, …).

1 Preexec hook

To enable the hooks, the user first has to load the add-zsh-hook function. To achieve that goal, paste the following line:

autoload -U add-zsh-hook

Once it is done, we are able to add and remove a function from a hook. For our case, we suppose we want to add the hook_function to the preexec hook. The following snippet shows how to do that.

hook_function()
{
  echo $1
  echo $2
  echo $3
}

add-zsh-hook preexec hook_function      # Add it to the preexec hook.
# add-zsh-hook -d preexec hook_function # Remove it for this hook.

Adding and removing a function from a hook is done the same way for every hooks.

The preexec hook is ran each time a command is read by the shell and is about to be executed.

Each function run by the preexec hook receives three arguments. The first one is the line as it was written. The second line is the line with alias expanded and truncated with a certain size limit. The third line is the full line with alias expanded. This thread shows an example. For the macros module, I decided to use the third expression because I want to be able to use my aliases in my macros. But that depends of the application you want to write.

2 Interactive prompt

You know that, there is plenty of ways to customize your prompts in Zsh. I'll just present one of them today, some post about the same topic might follow.

What I present today is how to use your RPROMPT to print some information about what you want, and is actualized every time you enter a new command. It is easy to do, here is the first step:

setopt prompt_subst

Maybe you have recognized the beginning of what you have to add to your configuration file to make the zsh macros module working? Well done! Otherwise, it doesn't matter. So what does this little line? According to the man (man zshoptions): "If set, parameter expansion, command substitution and arithmetic expansion are performed in prompts".

Let's see what happens if we don't set this option:

$ msg="Hello"
$ RPROMPT="($msg)"
$                                       (Hello)
$ msg="Goodbye"                         (Hello)
$                                       (Hello)
$ echo $RPROMPT                         (Hello)
(Hello)
$                                       (Hello)

Pretty annoying right? In fact, the shell expands $msg before it is received by RPROMPT, so what happens is simple, it prints the value of what he receives: "Hello" literally. So, let's see what happens if we re-execute the same sequence of commands with the prompt_subst option set?

$ setopt prompt_subst
$ msg="Hello"
$ RPROMPT="($msg)"
$                                       (Hello)
$ msg="Goodbye"                         (Hello)
$                                       (Hello)
$ echo $RPROMPT                         (Hello)
(Hello)
$                                       (Hello)

Here is the most common error (I think) that leads your prompt not to expand your variables. The RPROMPT command doesn't know there is a variable to expand, and you have to prevent your shell to expand it by single-quoting your assignation. This way:

$ RPROMPT='($msg)'
$ echo $RPROMPT                         (Hello)
($msg)
$ msg="Goodbye"                         (Hello)
$                                       (Goodbye)

There is two things to be careful with when you want to have your prompt expanding some content: the option prompt_subst must be set, and the content of the variable RPROMPT contains the thing you want to be expanded each time (Think about single quoting it!).

Now let's see what we can do with it! If you are a module writer, you can use a variable as flag (as I did), or give a function that allows to get information about something (as it's done by zsh to allow user to get vcs information).

For getting your branch in your RPROMPT, I recommend you to read the answer of ko-dos which is very complete. If you just paste the code, it will work. But you know why it uses single quotes, and why there must be the prompt_subst option set. For the zstyle part, I didn't try to understand it. One day, I'll try :)

Let's see how get the time in your right prompt. First, how to get the time only when calling the date command? I read this post to find the right format. It is just date +%T. Now let's apply what we have learn:

$ RPROMPT='$(date +%T)'
$                                       (23:42:00)

Now you are just limited by your needs and by your imagination :) If you make your own custom prompt, please share it in comments. I hope you like it!

ZSH Macros

Today, I'll present a module for zsh that I wrote few days ago. The aim of this module is to provide a way to create easily temporary shell scripts, and save their favorites. If you are familiar with Emacs, and if you think about its macros, you're right! I designed this with the macro concept in mind.

I had the idea when working with people who aren't familiar with shell scripts, and who don't want to try it for helping them. The original example was a work-flow with a TODO to update regularly. And yet commands to do were not so difficult:

$ git add TODO
$ git commit -m "Update the TODO."
$ git stash
$ git pull --rebase
$ git stash pop
$ git push

In reality, my coworker didn't plan to stash nor pull, but since this is my story, I can change it a little! :)

I thought that it is easy to write a script to make that works. I just have to copy these lines and paste them in a script. But, on one hand I find this boring, on the other hand someone who is not interested in scripts will never do that. So I had to find a transparent way for the user to have the same result without having the feeling that he plays with scripts. It's here that came the idea to mimic the behavior of the Emacs macros.

Notice that even if the Emacs macros works on text, and the title of this post might be confusing, I don't want to write a tool to enhance the Emacs macro in the Zsh command Line Editor (zle), but I want a way to create simple scripts in a easy and fast way.

1 Zsh macros!

I first tried to create a Perl script that can create scripts, but I realized that there is too many drawbacks (no history, no completion…). So I found an alternative way, fully integrated in zsh: hooks. For people who doesn't know what are hooks: It is a set of functions called at a specific time that allow the user to run their own functions. It is useful for letting the user personalizing a software. As an example, I wrote a git hook to check the log message (I talked about it in a previous post). For this module, I use the preexec hook for achieving my goal. In this post, I'll present my module, why it can be useful for day-to-day usage, and how to use it. In some next posts, I'll show some useful tricks I had to use to make it works.

1.1 Why using it?

Because it allows you to save your time. It is easy to install, easy to learn and easy to use. I realized that sometimes I repeat the same sequence of lines several time. It ends up by having these lines concatenated in one line with a && between them. Pretty ugly, right? But because it is just repeated less than ten times, I don't want to write a script for that because it is faster for me to just reuse my zsh history. But with this module, you just type your sequence of command once, and then you just have to hit macro_execute to get it repeated. Personally, I have aliased this command to e. It is the fastest and cleanest way I know to repeat your work properly.

1.2 How to install it

Glad to see you here! You'll see that it is a good choice :) The first step to install it is to get it from my github (directory zsh_macros). Once it is done, you just have to source the file, in your shell to try it, or in your configuration file if you're sure that it will fit to your needs.

Some things to check: if you have already bound either <ctrl-x><(> or <ctrl-x><)>, I don't bind anything (I don't want to mess up your own configuration!). These bindings are the same than under Emacs. Feel free to adapt these bindings to your own wishes!

You then have to add something in your configuration file:

setopt prompt_subst
RPROMPT='$(zmacros_info_wrapper)'

The first line allows the prompt to perform command substitution in prompts. The second one set your RPROMPT to the value of zmacros_info_wrapper that allows you to know the status of your macro. If you have already assigned something to your RPROMPT, you could simply add it to it.

Once this is done, everything should work fine. If this is not the case, you can either send me a bug report or send me a patch. Now, let's see how use this module.

1.3 How to use zsh macros?

In this part, I assume the bindings are the one originally provided. I think a screenshot may help to figure out how it looks like, I first run it in the shell, to show what you have to do, and what are the result. Between <> are represent the keyboard macros. Do not write it out :).

$ echo foo
foo
$ <ctrl-x><(> echo bar
bar
$ echo baz
baz
$ <ctrl-x><)> e
bar
baz

This screenshot shows how it appears for the user:

The flag on the right (<R$id>) appears right after you type <ctrl-x><(>, and disappear right after you type <ctrl-x><)>. Pretty easy right?

Note that if you don't like key-bindings (Are you a Vim user?), you can call macro-record and macro-end and you'll get the same effects.

Let's go a little deeper: you can have several macro. This module doesn't support nested macros in a same shell, but you can make as many macros as you want. Each macro is associated with an id. This is what is printed on the flag after the R in the prompt. You can run macro-execute with an optional argument that corresponds to the id of the macro you want to run. By default it's the last recorded. Notice that each script has its own file, and there is a master file that track each of them. To add and execute macros, we read and write on this file in /tmp. This way has its advantages and its drawbacks. We have concurrency problems, but since a real user can't make several things in the same time, that should not be a real problem.

The advantages are that a macro recorded in a shell can be used in another one, and you can register two macros at the same time, because the only access to the main file is made when you call macro-record. So recording two macros in two different shells is fine.

All your scripts live as long as your /tmp is not cleaned. If you want to keep a macro for a longer use, it is possible. You just have to call macro-name that will take the macro you asked for (if you give no id, the last one is considered), and copy it in the directory ZMACROS_DIRECTORY. You can set it at the top of the file macros.zsh. Maybe it is a good idea to add this directory to your path, it will allow you call these new functions simply.

This is the features available in this first version of the zsh macros. I planned to add some new ones, but if you have any request, comment, or anything, feel free to comment this post! I'd like to know what would be helpful and what you think about this module.

Debugging C++ (Part 3): dmesg

Welcome in the third post of this series about debugging C++. In here, I will talk about something less usual because it allows to debug after the crash of the program, this method use dmesg. I just present the case where we work with several libraries and your program crashes without any clue on which library is responsible of this, nor how to reproduce this behavior.

1 dmesg

I heard about dmesg when reading the tsuna's blog. Unfortunately I don't have any competence (for now) for reading assembler. But the fact that we can discover the name of the faulty function is helpful. I had to use this when I worked on a robot during my internship (a next post will present that). We worked with libraries and this is what shows my post.

On a robot there is a lot of parameters coming from the miscellaneous sensors, and the execution of the same program depends on a lot of parameters. So it is really hard to reproduce a bug. If the nice "segmentation fault" message appears, how can you debug that? Considering that you can't run valgrind, and running gdb is painful.

dmesg was my solution. I wrote a shell script to make the computation for me. Let's start by creating a dummy library which exports one function that segfault if a null pointer is given, and let be sadistic, we will call it with nullptr.

// file: libprint.hh
#ifndef TMP_LIBPRINT_HH_
# define TMP_LIBPRINT_HH_

int dereference(int* t);

#endif // !TMP_LIBPRINT_HH_


// file: libprint.cc
#include "libprint.hh"

int dereference(int* t)
{
  return *t;
}

// file: main.cc
#include "libprint.hh"

int main()
{
  return dereference(nullptr);
}

We create a libprint.so that contains the dereference function. And we compile the file main into a binary print linked with this library. And oh, surprise! Segmentation fault. Let's start the hunting. We call dmesg, and look at the last line:

[184608.332284] print[31332]: segfault at 0 ip b772e422 sp bf8ad218 error 4 in libprint.so[b772e000+1000]

We need two information: the name of the library that contains the bug, and the address of the faulty instruction in this library. To get the name of the library, we have to take the last field and to remove the part into []. To have the address of the faulty instruction we have to take the value of the instruction pointer (ip), and the value before the + in the last field. And we just have to subtract the value of the second value to the value of ip. If you are wondering why subtracting these two values to know the address of the ip in the library a draw may help.

I hope the picture helped, in fact, this subtraction removes the offset corresponding to the position of the library (address).

The question is how to make this process automatically? First, we can make the assumption that we always run dmesg right after the error, so we can suppose that we can make a call to tail to keep only the last line. But sometimes this assumption isn't correct, so our solution must be able to get a value given in argument. In here we use the shell default value assignment. As a little remainder:

output=$1
output=${output:="default value"}

If an argument is given, output will be equal to its value, otherwise it will be equal to "default value". So we can use it to decide whether we use the first argument of the program or directly call dmesg.

The part of the message before the colon is useless, so we can remove it. Then we have to get the value of the fifth field to get the value associated to ip, and we have to get the last field.

The name of the library and the address where it is mapped in the memory lie in the last field. So we have to cut it in two and we can get the needed information.

All these operations can be made by using only awk and sed.

Once we have the two addresses we just have to make the operation. We use the builtin system of the shell to make the subtract. Beware, they are in hexadecimal! So we must prefix the value by 0x to tell the base to the shell. Now we have the result (in decimal), we want it converted into hexadecimal, we use bc. It is a tool for making numeric computations. And we are grateful, there is a way to make it convert a number from a base to another. The syntax is simple, you have to set the variable obase to 16 (default value is 10). And that's all, remember to append the 0x before the address, because bc won't.

Here is the complete script:

#! /bin/sh

output=$1
output=${output:=`dmesg | tail -1`}
output=`echo $output | sed -e 's/.*: //'`

first=`echo $output | awk '{ print $5; }'`
second=`echo $output | awk '{print $11; }'`

library=`echo $second | sed -e 's/\[.*//'`
second=`echo $second | sed -e 's/.*\[//' -e 's/\+.*//'`

address=`echo $((0x$first - 0x$second))`
address=`echo "obase=16; $address" | bc`

echo "Segmentation fault in $library at: 0x$address."

And the way to use it is simple, just run it just after a segmentation fault when working with a library. Here is what it says about our case.

$ ./dmesg.sh
Segmentation fault in libprint.so at: 0x422.

And now, just run gdb like this (it is how I get with my libprint.so example):

$ gdb libprint.so
...
(gdb) disass 0x422
Dump of assembler code for function _Z11dereferencePi:
   0x0000041c <+0>:     push   %ebp
   0x0000041d <+1>:     mov    %esp,%ebp
   0x0000041f <+3>:     mov    0x8(%ebp),%eax
   0x00000422 <+6>:     mov    (%eax),%eax
   0x00000424 <+8>:     pop    %ebp
   0x00000425 <+9>:     ret
End of assembler dump.
(gdb) ...

If you are fluent with assembler you could read it, or use the meta data given by gdb: "Z11dereferencePi". Oops, I realized that I have forgot to use "-g" when compiling. Not important: we have a mangled symbol. We can use one of the method presented in one of my previous post. And voila, we know that our mistake is in the function dereference(int*). Pretty good when, without this method I was unable to know where it fails, why, and in the impossibility to reproduce it since there is too much parameters. I don't know how I would have done without this method.

I put this script on my github account, so if you want to fork it to enhance it, it is possible.

Hope you liked it!

How to use git to avoid writing ChangeLog by hand?

The standard GNU defines what must be a ChangeLog file (see:
http://www.gnu.org/prep/standards/html_node/Change-Logs.html). The
main goal of this is to be able to track bugs, and to understand the
history of a project.

1 Why keeping a ChangeLog?

In the past, we must keep a ChangeLog file for each project, since
there was no tool able to give all the history in every condition.
I am too young to know the work flow with CVS and other tool. I
learn the control version with SVN. But to have access to all the
history of a project, we'll need to be connected. And it is long.

Now we have git (or Mercurial, but I don't know this one), which are
distributed system, and they allow to keep all the history of a project
in local. So, why should we keep a ChangeLog file?

Pros

• When the project is released, the `git log` is not accessible.
  
• There is copyright issue in free software.
  
• It is easy to write a good ChangeLog with Emacs (and I'm sure it
  is easy with vim too).
  

Cons

• There is several tools to generate a ChangeLog file with the output
  of git log. I think about the tool 'gitlog-to-changelog' from the
  gnulib project (see: http://www.gnu.org/software/gnulib/).
  
• It is common, when playing with branches and rebasing a lot, to have
  conflict only in the ChangeLog file.
  

By generating the ChangeLog when making a release (or an archive), we
solve the problem of the history and the copyright. We can use a
ChangeLog file, not in the repository (maybe it is a good idea to put
the "ChangeLog" in the '.gitignore'), to write the log message, and
then we can use a little script to take the first entry and give it to
git.

For example, a simple function like this can do the trick:

commit()
{
   [[ ! -f ChangeLog ]] && {
   echo 'no ChangeLog in current directory' >&2
   return 1
   }

   git commit -m "`sed'1d;/^....-..-../Q;s/^\t//;' ChangeLog`" "$@"
}

This script is highly enhanceable. This is just an idea of what could
be the script I am talking about.

2 Using Emacs to write the ChangeLog

Now let's talk about the work flow, and the use of Emacs for writing
the log. Let's suppose we have a ChangeLog file at the root of the
project. The main idea is each time you modify something in a file,
you hit "C-x 4 a", and its open the ChangeLog, and add an entry
(which follows the GNU Coding Standard), you just have to write the
meaning of your change.

Before committing, think about add a one-line summary !

There is a little problem with Emacs at this level. In the past, the
common work flow was one commit by day. And the One True Editor
follows this standard. So there is a way to bypass this, an option
allows the function behind "C-x 4 a" to create a new entry. But it
does it each time, and this is not what we want. So I create a little
wrapper around this. Here is the function:

(defun new-changelog-entry()
  (interactive)
  (setq add-log-always-start-new-record t)
  (add-change-log-entry-other-window)
  (setq add-log-always-start-new-record nil))

; "C-x 5 a" runs new-changelog-entry.
(global-set-key "5a" (quote new-changelog-entry))

The idea is to set the option before calling, and unset it after. So,
only when we want to create a new entry, a new entry is created. :D

3 Be sure the log follows a good format

In the aim to be able to translate the output of `git log` into a
GNU standard compliant ChangeLog, the commit message must follow
a strict format. So, how to achieve this goal?

Git provides several kind of hooks. A hook is a script called when a
specific operation occurs. There is a lot of source on the web to know
what is a hook. Here I'll talk about the use of a script (developed by
me and one of my teacher) for solving the format of the git log.

It is a script which can be run server-side or client-side. You can
find this script here:
https://github.com/Enki-Prog/tools/blob/master/git/update. Here I will
talk about the problem of getting all the commit between two push. And
the way to know easily the file modified when committing.

The strategy we applied, is to authorize any kind of log in a personal
branch (`pseudo/feature'), and to reject a push when it is not (either
a `candidates/feature' or a branch with no `/') and don't follow the
format.

There are several thing checked, and it is shared by the two way to
call this script. The explanation above are talking about the way
to get the commits, and the information to be able to check.

The way we check after, is less interesting I think, because if you
read this article, maybe it is because you want to develop yourself
this kind of tools. And you just need the way to don't have to look
a lot on the web how to make this, all the information you need are
here or at worst on the script.

3.1 Server-side

We receive three arguments: the ref name, the old revision, and the
new revision.

1. If the new revision is a null sha1, it means it is a branch deletion.
   So, nothing to do here.
   
2. If the old revision is a null sha1, it is a branch creation.
   

   In this case, to get the new revision, the command to get the
   commits, we need to call:
   

   
   

   git rev-parse --not $otherbranches | git rev-list --stdin newrev
   

   
   
3. In the other case, we need to replace newrev by "oldrev..newrev".
   

To get the `$otherbranches' the command is:

git for-each-ref --format='%(refname)' refs/heads |
  grep -F -x -v $refname |
  grep -x 'refs/heads/\(candidates/.*\|[^/]*\)'

The first line gets the list of branches. The second filters out the
current branch. And the last one, keeps only the one which are non
personal branch.

3.2 Client-side

In this case, we have only one argument: the path to the temporary
file which contains the log which will be tested. In this case it is
easier, because there is only one commit. The question is, how to get
the list of modified file? There is several way, but the one I found
the simpler, is to make:

git status --porcelain

The output is simple: Two characters, and the filename (eventually two,
in the case of a `git mv`). If the first character is not a ? or a space,
the file is in the index and ready to be committed.

4 Conclusion

We have talk about the question "should I keep a ChangeLog in my
project?". And I developed on how to make this change in a good way.
Thanks to git, Emacs, a tool to check if the log is correct and
`gitlog-to-changelog'. Obviously, this is the way I choose for me,
and each part I present can be switched.

Feel free to leave a comment with your opinion and/or your suggestion :)