Debugging C++ (Part 2): Valgrind and gdb

This article is the second post of the "Debugging C++" series. This post is an introduction of Valgrind and gdb. It is intended for people that don't know these because it starts from the beginning and gives some links to the documentation. In the valgrind part, I present the notion of "definitely lost", "indirectly lost", etc. If you are already familiar with these notions, I invite you to go through the second part of this post about gdb. In this second part I start by briefly presenting what is gdb and what it is useful. But the main interest of this section is that I present the notion of reverse-debugging, which allows you to run your program backward. I also present a useful trick when you want to debug a code you wrote (and you can modify) inside an unknown environment (for example, imagine you want to debug your (buggy) malloc implementation and you test it with ls).

Before starting tools like Valgrind or gdb, think about compiling your code in debug mode, and not using optimizations. Otherwise you'll get trouble to debug it. For example with gdb, your cursor will jump from one line to another without following the linearity of the source code. This can be disappointing, this is due to that.

1 Valgrind

Valgrind is a set of tools that contains a memory checker (memcheck), a profiler (users can use two modules: callgrind and cachegrind), a tool for checking the memory consumption (massif), a synchronisation checker (Helgrind). The tools I use the most in this list is Memcheck, and this is all I will talk about in this post. I'm sure there will be other posts on this list of tools in the future. This is just a brief introduction. If you already know valgrind, you should skip this part.

Memcheck helps you to detect any memory corruption. How it looks like? Let's assume you have a program like this:

#include <vector>

int main(int argc, char *argv[])
{
  std::vector<int> v;
  v.reserve(2);

  v[2] = 4;
}

The call to reserve allocates the space for two integers. But we try to write in a unallocated address. So it is a mistake from the programmer, and even if in this example it is trivial that there is a mistake, sometimes it is not, because it is hidden in a lot of code. So, detecting this is not so simple. That's where Memcheck is helpful. Here is what he says about this snippet.

==4641== Invalid write of size 4
==4641==    at 0x40099A: main (test.cc:8)
==4641==  Address 0x5955048 is 0 bytes after a block of size 8 alloc d
==4641==    at 0x4C286E7: operator new(unsigned long)\
 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4641==    by 0x400FB3: __gnu_cxx::new_allocator<int>::\
 allocate(unsigned long, void const*) (new_allocator.h:94)
==4641==    by 0x400ECA: std::_Vector_base<int, std::allocator<int> >::\
 _M_allocate(unsigned long) (in /tmp/aout)
==4641==    by 0x400D3B: int* std::vector<int, std::allocator<int> >::\
 _M_allocate_and_copy<std::move_iterator<int*> >(unsigned long, \
 std::move_iterator<int*>, std::move_iterator<int*>) (stl_vector.h:1109)
==4641==    by 0x400AC8: std::vector<int, std::allocator<int> >::\
 reserve(unsigned long) (vector.tcc:76)
==4641==    by 0x400988: main (test.cc:6)

4641 corresponds to the PID of the process. Every line coming from valgrind is formatted like this: ==PID==. Line beginning by a space are just the continuation of the line above. It was just for having the output of valgrind fitting in the page.

Remember to use "-g" when you compile your program, otherwise you'll get the name of the binary instead of the name of the file and the line number.

It starts by giving the name of the violation: "Invalid write" and tells us how many bytes we violate. Here this is 4 (sizeof(int)). Then he tells us where we have made a mistake, and then he shows what is the mistake. We have written after an allocated area. And then he shows the stack of the call that led to allocate this area. It starts by a call to reserve in the main line 6, and so on.

As you can see, the message is clear and it helps finding this kind of mistakes easily. This is a valuable tool for writing bug-free software. Every of your program should work without any error in valgrind. Because even if it doesn't create an error directly, it can lead to very weird error later and this is called (in my school at least^^) a mystical error. Because there is no clue that could help (without valgrind).

This is a simple example just to see how powerful is this tool. Another common use of Memcheck is its ability to detect memory leaks. As an example:

int main()
{
  int* p = new int;

  p = nullptr;
}

This is a trivial case of a memory leak because after the affectation of p to nullptr, there is no more pointer to the area allocated with new. It is lost. This is trivial to detect it, but that could be less trivial. So once again, we can rely on Memcheck to warn us. Let's see what he has to say about this:

==4736== HEAP SUMMARY:
==4736==     in use at exit: 4 bytes in 1 blocks
==4736==   total heap usage: 1 allocs, 0 frees, 4 bytes allocated
==4736==
==4736== 4 bytes in 1 blocks are definitely lost in loss record 1 of 1
==4736==    at 0x4C286E7: operator new(unsigned long) (in \
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4736==    by 0x40060D: main (tests.cc:3)
==4736==
==4736== LEAK SUMMARY:
==4736==    definitely lost: 4 bytes in 1 blocks
==4736==    indirectly lost: 0 bytes in 0 blocks
==4736==      possibly lost: 0 bytes in 0 blocks
==4736==    still reachable: 0 bytes in 0 blocks
==4736==         suppressed: 0 bytes in 0 blocks

I just show the interesting part of the output of valgrind. To get this, I ran valgrind –leak-check=full <program_name>. We start by reading the heap summary, which says that when we exit, 4 bytes are still in use. It indicates the number of allocation and the number of deallocation. We see that the two numbers are not equal, so there is a problem. There is more allocation than free, it's a leak.

At the bottom of the message we can see the kind of leak. There are four kinds of leaks. The full explanation can be found in the documentation, in the section 4.2.7. A short explanation follows.

AA and BB are heap blocks, and C the set of pointer of your program (in reality it is more than that. See the section 4.2.7 cited above). And consider an arrow represent that there is at least one pointer to the first byte of the allocated area, and an arrow and a `?' represents a pointer to a byte inside the allocated area and not the first (this is called interior-pointer in the link given above). No arrow means that the heap block is unreachable. These are simplified examples to understand the concept behind these things.

Once we have understood what represents these names, we can look at the message in the middle that gives the location of each memory blocks that leak. It says definitely lost. So we just override the address of this area, and it appears line 3. Once we know the supposed reason of this leak and its position in our source file, it is easy to fix it.

This is the power of Memcheck. This is the first thing I run when I have a bug. Because even if the error he can reveal is not responsible of the bug, it will be annoying in the future.

A important note about valgrind, since it use its own allocator and deallocator, some bugs may disappear when using Memcheck. It happens to me (at least) once, when I had the output of my program that depends on the state of the computer (state of the heap in fact), which led my algorithm to have different results if I run it several time consecutively. This is generally due to a memory corruption or something like that, so I used Memcheck. And under Memcheck, the program starts to act normally.

An interesting post about valgrind that talks about can be found here.

2 gdb

A second tool I use when I still have a bug after using valgrind is gdb (GNU Debugger).

gdb is a debugger that allows you to walk through your code at run-time and see how the lines you (or your colleagues) wrote influence the program and detect bugs.

I don't want to write a basic tutorial to gdb since there are really tons of them on the Internet. I'd prefer talking about two things that can be useful and not known by everyone. The first one is backward debugging. The second one is a trick to debug a library when you can't run gdb on the program that uses the library. Baptiste Afsa, one of my teacher at Epita, gave me this trick.

2.1 Backward debugging

Why backward debugging? Simply because I facepalmed myself too many time after going too far in my debugging process. I mean going after the critical point after running through trivial code slowly for 5 minutes and being forced to restart the whole thing.

This feature was introduced in 2009 with gdb version 7.0. Documentation is here. To be able to enable this feature, you need to tell gdb to record the state of your program. The official tutorial is here. So I don't have to explain that in details (No I'm not lazy! :) but I don't like duplication and I don't like duplication). I recommend to read the tutorial!

But the thing is that recording introduces an overhead when you use next, continue or whatever. This can slow down the debugging process, but I think it can really help the programmer.

Give it a try, it is helpful.

2.2 The infinite loop trick

This tips allows you to debug the code of a library call by a program that you have difficulties to run with gdb. The case where I use it is when we had to code malloc (free, realloc, calloc too) at school. And to check if it works we used LD_PRELOAD to make binary use our version of malloc instead of the standard version.

Let's assume we run ls with our malloc, how would you debug it?

$ gdb ls
...
Reading symbols from /bin/ls...(no debugging symbols found)...done.
(gdb) break main
Function "main" not defined.

Well… Seems hard right? The solution given by my teacher was to use an infinite loop. How it works:

#include <iostream>

int main()
{
  bool cond = true;

  while (cond)
    continue;

  std::cout << "Out of the infinite loop" << std::endl;
  // Stuff
}

The trick was to add this snippet at the beginning of the code of our function, and to run the program. For example, compile the code above (don't forget -ggdb). This is an infinite loop, yay!

gdb can attach to a running process with its PID, and is able to set the value of a variable when debugging. So the trick is just:

$ pgrep <program_name>
<pid>
$ gdb attach <pid>
...
main () at tests.cc:8
8           continue;
(gdb) set var cond = false
(gdb) n
7         while (cond)
(gdb) n
10        std::cout << "Out of the infinite loop" << std::endl;
(gdb)

By setting the variable cond to false, we are out of the infinite loop, and we have the hand on the program to debug as we want. Impressive right?

I think this trick is useful for desperate situation like the one described above.

This was two great features I wanted to share with you. Do you have features that you want to share?

Debugging C++ (Part 1): How to write less bugs

Hi all! I planed to make a post about some different methods to debug a C++ program. And I realized that it will be a very long post, so I split it into 4 parts. The first part is about some methods to avoid dummy bugs notably using two new C++11 features. The second is a presentation of valgrind and gdb. I include in the gdb part an introduction to the reverse-debugging that consists in running the program backward. The third part is an explanation of the usage of dmesg to find an instruction in a library that leads to a segmentation fault. One of the advantage is that it works after the crash, and don't need to restart the program. The fourth and last part is about the print method. I present a way to make this method pleasant to use and easy to maintain.

This is far from being an exhaustive list of debugging methods, just some of my favorites. You are invited to share yours in comments! :-)

This first post presents the importance of using warnings when compiling, assert to verify the coherence of the program, and miscellaneous things introduced by C++11.

1 Warnings

The first thing to do to avoid stupid bugs is to think before writing any piece of code. It can be hard sometimes, but it's totally worth it.

My global philosophy about programming is that I want my computer to insult me whenever he can. I want a compiler able to detect as many errors as possible.

So, the thing to do in the aim to make the compiler as hard as possible, is to enable warnings. Personally, on g++ I always use -W, -Wall, -Wextra and -Werror for changing all the warnings into errors. This can save some hours of debugging. Let's see an example of a buggy code that compiles without warning, but with enable warnings it won't and it is great!

int i = -42;
unsigned int j = 51;

if (i > j)
  {
     // Bug found.
  }

It can be disappointing that i > j is evaluated as true. It is due to an implicit conversion. The i once compared with an unsigned int is converted into a unsigned int equal to UINT_MAX - 41. So this is really easy to make this error when the type are declared too early and you forgot what is the type of i and j. Warnings are just mandatory! I hope this little example is enough to convince you. I'm sure there are several hundred of examples like this one, and you just have to run through the net to find out another examples.

2 Assert

A good practice is to use the macro assert available in the header cassert. This is a macro that evaluates its content and stops the program if its content is evaluated to false. If you define NDEBUG (the common way is to pass the -DNDEBUG option to g++, -D allows to define a macro), the code inside the parenthesis of the macro isn't evaluated.

The main interest of assert is that it can be a good checker for preconditions or postconditions. Beware, you must not use it as a way to manage run time error. It is here to verify all along of your development that you are not receiving something weird. If you use well assert, it must stop the flow of your program before it starts acting crazily. By making this, you ensure looking at the good spot for finding the source of the problem, and not to a side effect that occurs 20 functions later. This can reduce considerably the debugging time.

As said above, the code between the parenthesis isn't evaluated in release mode. So a bad use of assert would be to put real code in it. Because once released, this code will not be ran. It is also its advantage. Checking all these preconditions can introduce an overhead, but you don't have to worry about it in release mode since this is like this code never exists.

As a little conclusion, if you don't already use assert, start now! :) It can change a lot of things and it has already saved a lot of debugging hours for me. I hope it will be the same for you!

3 Miscellaneous

3.1 Preventing Narrowing

Now I will give some little tips which can help. There are a lot of tips like this. Once again, I invite you to leave your own tips in the comments!

A common problem in C or C++ is narrowing. Preventing this is an addition of the C++11, which can prevent a lot of bugs. As an example:

void doit(int);

int main()
{
  float i = 4.2;

  doit(i); // Huum... A bug hard that could be hard to find.
  doit({i}); // warning: narrowing conversion of 'i' from
             // 'float' to 'int' inside { } [-Wnarrowing]
}

This examples shows how it can help to avoid some kind of bugs. I recommend using it around all the variables you want to protect. These situations happens, and why not use the language to help you to not losing your time?

3.2 nullptr

It is also important to use strong typed variable. It helps the compiler to help you! Once again, C++11 comes with a strongly type null pointer nullptr. NULL is just 0 (see Stroustrup FAQ). And it can lead to bugs related to the dispatch on overloaded function.

void print(long int i) {  std::cout << "long int: " << i << std::endl; }

void print(int* i)     {  std::cout << "pointer" << std::endl;         }

int main()
{
  long int i = 51;

  print(i);       // prints "long int: 51"
  print(NULL);    // Raises a compile-time warning and prints "long int: 0"
  print(nullptr); // prints "pointer"
}

The warning is "passing NULL to non-pointer argument 1 of 'void print(long int)'". Hopefully there is a warning in this case because this is not the wanted comportment. The introduction of nullptr allows to represent the concept of a null pointer and to have it strongly and correctly typed. I think it is a good idea to use it instead of the NULL or 0.

3.3 Yoda Condition

I use this name after reading this very funny post about new programming jargon. This goal is for people who makes typo when they write like writing = instead of ==. I have to admit, I have rarely something like this written in my code since I don't use magic number (constant values written in the source in the middle of the code). But sometimes it can help. Here is an example:

int main()
{
  int i = 51;

  if (i = 51)
    std::cout << "Oops" << std::endl;

  if (51 = i)
    std::cout << "Thanks g++!" << std::endl;
}

Since some people want to write assignment in their conditions, the compiler can't warn about this. So you have to make it scream by helping him. In the second if we get an error "error: lvalue required as left operand of assignment".

That's all for the little tips, I hope you see why being drastic with yourself can help you. Writing these asserts is longer than not writing them because you have to think to all the precondition needed etc. But I can assure you that you are so happy when you see your program crash because of an assert and not with a segmentation fault or some crappy things like that. About the warnings, at the first glance, it seems annoying to be warns about everything, but programming is made of little details too. So use it! :)

For the miscellaneous tips, this is just little habits to take that can improve the work flow by reducing little mistakes. The last one is more a funny thing than a strong guideline as are using {} to prevent narrowing and nullptr to help the compiler by saying that we use a pointer.

Don't hesitate to post your own tips in comments ;)

C++11: Vector Improved, How It Works?

C++11 comes with several interesting things, and in this post, we will talk a little about a new method in std::vector that comes with the new standard. This method can lead to improve the performance of your programs. This post is divided in two: The first part explains why it is good to have this new method (emplace_back) and how to use it, and the second part will try to make the way it works clear. To achieve this goal, we have to go through several new things (variadic template, constexpr, argument forwarding) that we explain a little to understand the whole thing.

In the previous standard, sometimes you had to create an array of let's say Point3D (or whatever) in a loop. Let's suppose we know the number of elements we have to put in the vector, we first show the slowest version I think about, and then we show a more optimized version thanks to C++11 standard.

1 A Case Study

struct Point3D
{
  Point3D(int xx, int yy, int zz)
    : x {xx}
    , y {yy}
    , z {zz}
  {
    std::cout << "Cstor" << std::endl;
  }

  Point3D(const Point3D& p)
    : x {p.x}
    , y {p.y}
    , z {p.z}
  {
    std::cout << "Copy Cstor" << std::endl;
  }

  int x;
  int y;
  int z;
};

int main()
{
  std::vector<Point3D> v;

  for (unsigned int i = 0; i < 10; ++i)
    {
      v.push_back(Point3D {i, i - 69, i + 42});
    }

  for (unsigned int i = 0; i < 10; ++i)
    {
      std::cout << v[i].x << std::endl;
    }
}

What we want in this program, is to have only 10 calls to the constructor because we just want ten objects, so our requirement seems legit right? But if we run this program, we can see 10 calls to the first constructor and 25 to the copy constructor. This is huge! There are two reasons for these copies:

• Vectors are dynamic arrays. And each time it reaches its limit, it doubles its size. So all the elements are copied at each reallocation. It starts with a size 1, then has to double its size, and the same for 2, 4 and 8. If we add these numbers, we have 15 copies. These can be deleted by using the "reserve" method since we known the number of elements. By this call we avoid the reallocation and the copies.
• The problem of who is responsible of the destruction of which object is a complex one. The STL handles this by copying the object it takes in their containers and to destroy them when the container is destroyed. This is why we still have 20 copies and not 10. But with C++11 comes an emplace_back method (emplace exists too, but we focus on the push_back dual here). This method removes the copy done by the push_back method. For a user, the only changes to make is to replace the call to push_back by a call to emplace_back. The arguments to give to the new method are the arguments to give to the constructor:

v.emplace_back(i, i - 69, i + 42);

2 How It Works

Now, let's see what are the changes done for being able to make this emplace_back method. For this post I have used the glibcxx version 4.7.1 to discover the changes.

Before going into the code, we have to present variadic templates and its power, the forwarding parameters concept, and how to use it. Then, we show the difference between push_back and emplace_back.

2.1 Variadic Template and Forwarding

Forwarding parameters is a way to take all the arguments received by a function, and to resend it as it comes. This comes with variadic templates. Variadic templates allow to pass an undefined number of arguments to a function/method. It allows to have a program which looks functional (a Head and the Rest). The code that follows combines two new features of the C++11, constexpr and variadic parameters. A little word about constexpr: it allows to make computation at compile time if they are possible (there are several conditions to respect for that, but for now, we just assume that they are respected). In this example, we find the min of a list of argument at compile time (thanks to constexpr). This is a generic version that works only with integers but works on 2, 3, 4, … n arguments.

constexpr
int min(int n, int p)
{
  return n < p ? n : p;
}

template<typename... Args>
constexpr
int min(int n, Args... args)
{
    return min(n, min(args...));
}

int main(int argc, char *argv[])
{
  static_assert(min(4, 5, 6, 42, 7, 3, 6) == 3, "min is incorrect");
}

The recursion mechanism is classical, but what I would have implemented with a vector in the past (in a run-time version) or template recursion (in a compile-time version) is fully expressible with the concept of variadic template, and it is enjoyable, because it is kind of beautiful. It allows to be computed at compile-time if all the arguments are deductible at this time, or simply computed at run-time. It is also invisible for the user.

It is important to note that the arguments are passed by copy. If we wanted to pass them by reference, the game would be harder… And to be honest, at this time, I don't know how I should do. It seems more complex to handle the recursion because it implies to implement different combination of the arguments ( int&, int&&; int&&, int&; …). There was a proposal to make it as a part of the standard library (proposition n772), or with initializer_list. The proposal shows it is easier to implement and faster with the second method. We keep this min implementation as a simple example to understand variadic templates. Any proposition of a fully generic working version (using constexpr and variadic template) of this is encouraged! A solution might be to use std::forward since it seems to be made for handling the forwarding, but my first tries were not successful.

Now you have seen why I think the variadic template are good to play with, let's talk about forwarding arguments. Forwarding is made by the std::forward function from the `utility' header (code can be found in `bits/move.h'). It forwards the arguments exactly as they are received (more information in the man). Here is a little example of what makes std::forward in the code:

#include <iostream>
#include <utility>

struct Item
{
  Item(): value{0} {}

  Item(const Item& p){ std::cout << "Copy" << std::endl; }

  int value;
};

void take_args(int& a, int&& b, int c, Item& f)
{
  std::cout << a << " - " << b << " - " << c
            << " - " <<  f.value << std::endl;
}

template <typename... Args>
void call_take_args(Args&&... args)
{
  take_args(std::forward<Args>(args)...);
}

int main()
{
  Item f;
  int i = 2;
  call_take_args(i, 4, 5, f);
}
// The program outputs "2 - 4 - 5 - 0".

If we remove the reference in the last take_args argument, "Copy" is also printed. We can see that the program won't compile if we remove a `&' for the second argument, or if we add an extra one to the first. This is because std::forward keeps the r/l-valueness of the argument received. How are they able to know this? Let's take a look at the source of this function (version 4.7.1 of glibcxx):

template<typename _Tp>
constexpr _Tp&&
forward(typename std::remove_reference<_Tp>::type& __t) noexcept
{ return static_cast<_Tp&&>(__t); }

template<typename _Tp>
constexpr _Tp&&
forward(typename std::remove_reference<_Tp>::type&& __t) noexcept
{
  static_assert(!std::is_lvalue_reference<_Tp>::value, "template argument"
                " substituting _Tp is an lvalue reference type");
  return static_cast<_Tp&&>(__t);
}

The first version of the function takes a lvalue and the second a rvalue. The static_assert just checks if the specialization works as expected. This specialization works because std::remove_reference takes a type `T' (possibly U, U&, U&&), and the `type' is a typedef of T. So we are able to manage a use case where we give as parameter T, T& or T&& the same way. As a result we have two functions with the following signature: forward(T& __t) and forward(T&& __t). If the argument is a lvalue it goes in the first one, otherwise to the second. noexcept just helps the compiler to know that the function/method will not throw an exception or that the program should be stopped if an exception tries to escape from here.

Now let's take a look at what they return. The fact that both functions seem to return the same thing might look weird. We have the feeling that they return the same type: `T&&'. But in fact, `T' could be a `U&' or a `U&&', and C++11 comes with a rule named reference collapsing rules (see this article which talk about it). This is simple:

• U& & => U&
• U&& & => U&
• U& && => U&
• U&& && => U&&

This looks like the `and' truth table where & is 0 and && is 1. A good way to remember I think. So: if _Tp is a U& (the first function), the returned object will be U& too, and if it was U&& it will be U&&. Which follows the rules of a perfect forwarding. Now we have understood the concept of the forwarding, and the way it is done, we can take a look at the code of push_back and emplace_back to know what makes the change.

2.2 Emplace_back and Push_back

Now we have all the tools in our hands to understand the difference between these two methods, we can just show the source code:

// Taken from bits/stl_vector.h
void
push_back(const value_type& __x)
{
   if (this->_M_impl._M_finish != this->_M_impl._M_end_of_storage)
     {
        _Alloc_traits::construct(this->_M_impl, this->_M_impl._M_finish,
                                 __x);
        ++this->_M_impl._M_finish;
     }
    else
#ifdef __GXX_EXPERIMENTAL_CXX0X__
      _M_emplace_back_aux(__x);
#else
      _M_insert_aux(end(), __x);
#endif
}

// Taken from bits/vector.tcc
template<typename... _Args>
void
vector<_Tp, _Alloc>::
emplace_back(_Args&&... __args)
{
   if (this->_M_impl._M_finish != this->_M_impl._M_end_of_storage)
     {
        _Alloc_traits::construct(this->_M_impl, this->_M_impl._M_finish,
                                 std::forward<_Args>(__args)...);
        ++this->_M_impl._M_finish;
     }
   else
       _M_emplace_back_aux(std::forward<_Args>(__args)...);
}

The main difference is the use of the std::forward. The macro __GXX_EXPERIMENTAL_CXX0X__ is defined when an option to activate C++11 is set. The _M_emplace_back_aux and _M_insert_aux are responsible of the reallocation when the number of available position (available means allocated and not already taken) is down to 0.

Since there is no other difference between the two methods, that means the interesting point is in the construct method (we just leave the *aux method for this post, it will take too much time to analyze them).

Before that, we have to present briefly the concept of putting `::' before a function/method name and the concept of using a placement new. If your are familiar with these concepts, you should skip these paragraphs and go directly after the code of the construct method.

It is possible for a class to override its operator new. This (static) method is responsible of the allocation of the object. If we put `::' before the operator new that means that we want to take the one in the global namespace and not the overridden one. The code below shows the difference between these two notation:

struct Foo
{
  Foo(int a)
    : value{a}
  {
  }

  static void* operator new(size_t)
  {
    return (void*)42; // g++ is able to detect if
                      // we return 0 and raises a warning.
  }

  int value;
};

int main(int argc, char *argv[])
{
  Foo* bad = new Foo(4); // segmentation fault.
  Foo* good = ::new Foo(4); // ok.
  return 0;
}

The operator new returns a pointer to an unallocated area, which leads to a segmentation fault when the constructor is called and tries to write the value of `a'. If we put `::' in front of new, this buggy operator new is not called, and there is no memory corruption.

Now, let's talk about placement new. This is just a way to use specific position in the memory, and this can be useful when working with memory pools, or when performance is needed: allocation and deallocation are expensive. Avoiding this can speed up the program. The syntax is just: new(position), where position is a pointer to where the object must be. This is as simple as that.

Now let's close the parenthesis and look at the code of construct (this is not the construct which is directly called, but the other functions that calls this one are not interesting since the same path is followed by emplace_back and push_back):

// Taken from ext/new_allocator.h
template<typename _Up, typename... _Args>
void
construct(_Up* __p, _Args&&... __args)
{ ::new((void *)__p) _Up(std::forward<_Args>(__args)...); }

This version is the C++11 version, which manages the case where we copy, and the case where we construct. Let's start by understanding this line, they use the placement new because this is not up to the `new' to decide where put this object, since it must be put at the end of the vector.

There is a call to the constructor and the arguments received are forwarded. These arguments are the ones given to the emplace_back method. And then it is up to the overloading to make it works. If we are making a copy (we arrived here by push_back), the copy constructor is called. Otherwise, we call another constructor if it exists (if not, we get a compile-time error).

And that's all! To arrive here, we have to know variadic template, argument forwarding, constexpr, placement new, operator new overriding and reference collapsing rules. But at least, we understand what are the changes needed to be able to make this emplace_back method, and how the C++11 makes it possible. Thanks to the new standard for this improvement! :-)

PS: Thanks to Christopher Chedeau and Gregoire Bouchetoun for their comments and corrections about this article.

How To Demangle C++ Symbols

I am finally on holidays ! I was pretty busy these last months… Now I have some times for me, I am able to unstack a lot of stuff I wanted to share with the lost reader(s ?) of this blog. Let's start with a few things about C++ and its mangling.

The C++ mangles its symbols. But sometimes it is hard to reread these names. For example when you work with a lot of different libraries and when you get a "undefined reference to <hardly-readable>". So in this post, I'll show you how to decypher these symbols in two different ways on GNU/Linux.

void leave_a_comment()
{
  return;
}

void foo(int v)
{
  int this_blog_is_cool = v;

  if (this_blog_is_cool)
    leave_a_comment();
}

int main(int argc, char *argv[])
{
  foo(42);
}

Here is an example of a simple and useless C++ program, to show what makes the C++ with these function names. Let's assume that the source code above is in the file bar.cc.

$ gcc -c bar.cc
$ nm bar.o
00000000 T _Z15leave_a_commentv
00000005 T _Z3fooi
0000001e T main

It seems readable. _Z is reserved word for C++, the number after corresponds to the number of characters that composed the function name, for example here it is 3 for `foo' and 15 for `leave_acomment` It is then postfixed by the type of arguments. For the first function no argument, it is void (v). For the second one, it is an int (i). In this case it is easy to understand what corresponds to what. But it is enough to see what are our choices to demangle this identifiers.

I personally know two ways, (maybe there is more, thanks for reporting it ! :)). The first way is to use `nm' itself. It has a nice option named `-C' which allows to demangle the identifiers. If we use it, we get this results:

$ gcc -c bar.cc
$ nm -C bar.o
00000000 T leave_a_comment()
00000005 T foo(int)
0000001e T main

Which is far better right? In fact `-C' is a shortcut for the `–demangle' option. It can takes several style as input (see the man for more information). Warning, it seems that this option is not defined by the Single Unix Specification.

Another way is to use c++filt which is a tools from `GNU Binary Utilities'. It allows to demangle C++ symbols. It can either takes a mangled symbols in the command line, or read it from stdin. You don't have to filter yourself the output (for example the nm output), to make it works. You just have to give it all and he replaces what he has to. Here is an example:

$ gcc -c bar.cc
$ nm bar.o | c++filt
00000000 T leave_a_comment()
00000005 T foo(int)
0000001e T main

That's all folks ! I hope this will be useful ;)

How to use git to avoid writing ChangeLog by hand?

The standard GNU defines what must be a ChangeLog file (see:
http://www.gnu.org/prep/standards/html_node/Change-Logs.html). The
main goal of this is to be able to track bugs, and to understand the
history of a project.

1 Why keeping a ChangeLog?

In the past, we must keep a ChangeLog file for each project, since
there was no tool able to give all the history in every condition.
I am too young to know the work flow with CVS and other tool. I
learn the control version with SVN. But to have access to all the
history of a project, we'll need to be connected. And it is long.

Now we have git (or Mercurial, but I don't know this one), which are
distributed system, and they allow to keep all the history of a project
in local. So, why should we keep a ChangeLog file?

Pros

• When the project is released, the `git log` is not accessible.
  
• There is copyright issue in free software.
  
• It is easy to write a good ChangeLog with Emacs (and I'm sure it
  is easy with vim too).
  

Cons

• There is several tools to generate a ChangeLog file with the output
  of git log. I think about the tool 'gitlog-to-changelog' from the
  gnulib project (see: http://www.gnu.org/software/gnulib/).
  
• It is common, when playing with branches and rebasing a lot, to have
  conflict only in the ChangeLog file.
  

By generating the ChangeLog when making a release (or an archive), we
solve the problem of the history and the copyright. We can use a
ChangeLog file, not in the repository (maybe it is a good idea to put
the "ChangeLog" in the '.gitignore'), to write the log message, and
then we can use a little script to take the first entry and give it to
git.

For example, a simple function like this can do the trick:

commit()
{
   [[ ! -f ChangeLog ]] && {
   echo 'no ChangeLog in current directory' >&2
   return 1
   }

   git commit -m "`sed'1d;/^....-..-../Q;s/^\t//;' ChangeLog`" "$@"
}

This script is highly enhanceable. This is just an idea of what could
be the script I am talking about.

2 Using Emacs to write the ChangeLog

Now let's talk about the work flow, and the use of Emacs for writing
the log. Let's suppose we have a ChangeLog file at the root of the
project. The main idea is each time you modify something in a file,
you hit "C-x 4 a", and its open the ChangeLog, and add an entry
(which follows the GNU Coding Standard), you just have to write the
meaning of your change.

Before committing, think about add a one-line summary !

There is a little problem with Emacs at this level. In the past, the
common work flow was one commit by day. And the One True Editor
follows this standard. So there is a way to bypass this, an option
allows the function behind "C-x 4 a" to create a new entry. But it
does it each time, and this is not what we want. So I create a little
wrapper around this. Here is the function:

(defun new-changelog-entry()
  (interactive)
  (setq add-log-always-start-new-record t)
  (add-change-log-entry-other-window)
  (setq add-log-always-start-new-record nil))

; "C-x 5 a" runs new-changelog-entry.
(global-set-key "5a" (quote new-changelog-entry))

The idea is to set the option before calling, and unset it after. So,
only when we want to create a new entry, a new entry is created. :D

3 Be sure the log follows a good format

In the aim to be able to translate the output of `git log` into a
GNU standard compliant ChangeLog, the commit message must follow
a strict format. So, how to achieve this goal?

Git provides several kind of hooks. A hook is a script called when a
specific operation occurs. There is a lot of source on the web to know
what is a hook. Here I'll talk about the use of a script (developed by
me and one of my teacher) for solving the format of the git log.

It is a script which can be run server-side or client-side. You can
find this script here:
https://github.com/Enki-Prog/tools/blob/master/git/update. Here I will
talk about the problem of getting all the commit between two push. And
the way to know easily the file modified when committing.

The strategy we applied, is to authorize any kind of log in a personal
branch (`pseudo/feature'), and to reject a push when it is not (either
a `candidates/feature' or a branch with no `/') and don't follow the
format.

There are several thing checked, and it is shared by the two way to
call this script. The explanation above are talking about the way
to get the commits, and the information to be able to check.

The way we check after, is less interesting I think, because if you
read this article, maybe it is because you want to develop yourself
this kind of tools. And you just need the way to don't have to look
a lot on the web how to make this, all the information you need are
here or at worst on the script.

3.1 Server-side

We receive three arguments: the ref name, the old revision, and the
new revision.

1. If the new revision is a null sha1, it means it is a branch deletion.
   So, nothing to do here.
   
2. If the old revision is a null sha1, it is a branch creation.
   

   In this case, to get the new revision, the command to get the
   commits, we need to call:
   

   
   

   git rev-parse --not $otherbranches | git rev-list --stdin newrev
   

   
   
3. In the other case, we need to replace newrev by "oldrev..newrev".
   

To get the `$otherbranches' the command is:

git for-each-ref --format='%(refname)' refs/heads |
  grep -F -x -v $refname |
  grep -x 'refs/heads/\(candidates/.*\|[^/]*\)'

The first line gets the list of branches. The second filters out the
current branch. And the last one, keeps only the one which are non
personal branch.

3.2 Client-side

In this case, we have only one argument: the path to the temporary
file which contains the log which will be tested. In this case it is
easier, because there is only one commit. The question is, how to get
the list of modified file? There is several way, but the one I found
the simpler, is to make:

git status --porcelain

The output is simple: Two characters, and the filename (eventually two,
in the case of a `git mv`). If the first character is not a ? or a space,
the file is in the index and ready to be committed.

4 Conclusion

We have talk about the question "should I keep a ChangeLog in my
project?". And I developed on how to make this change in a good way.
Thanks to git, Emacs, a tool to check if the log is correct and
`gitlog-to-changelog'. Obviously, this is the way I choose for me,
and each part I present can be switched.

Feel free to leave a comment with your opinion and/or your suggestion :)

Hello World!

Welcome in my technical blog!

I'll talk here about several things:
- Projects I'm working on,
- Development tools I'm using and some useful tricks for them (emacs, git...)
- Development problem I have encountered, and I hope, the solution I have found to solve them,
- And other miscellaneous stuff about programming.

I hope you will find things that will be useful for you ;)