I have dedicated the last few days of work to face a problem that is getting bigger and bigger. Although we have a quite modular codebase, use interfaces whenever possible to hide implementation details and apply most of the recipes recommended in classic books, our linking times have started to be a problem in some of our libraries. Linking times is not the only problem. Our binary sizes are becoming really fat.

After a thoroughly study with the help of tools like Sizer, Symbol Sort (I strongly recommend reading the articles associated to this tool: 1, 2, 3, 4, 5, 6 and passing all them to your co-workers) we discovered lot of code bloating generated by improper template usage. The template is a powerful tool that can be easily misused although I am not against using it. We use template metaprogramming in lots of places like reflection, annotations, uri-like dependency properties, serialization and more places where it is worth it. The problem with templates is that normally declarations and definitions go in the same header file. With that structure a compiler is normally forced (although some compilers allow customization) to generate template instances in each obj file. Redundancy is eliminated in the linking phase, of course taking time. This is one of the faces of template bloating. The other face appears when you templatize functions when it is not strictly necessary. And you pay for it in the final size of your binaries.

Trying to solve all this, we applied a solution that although yet not standard seems to work in all compilers we tested: exporting explicit template instantiations. This technique applies when you have a template that is to be instantiated for only a few known types. Those instantiations are exported from a dll. This allows even hiding big functions in the .cpp without having to show them in the header. With this solution you have the best of both worlds: short functions can be inlined by the compiler and large functions instead of being instantiated in each compilation unit are referenced as an exported symbol. By the way, a special behavior of Visual Studio (to me, it is a bug) has the effect of non inlining functions being defined outside the class when exporting the template. We had to move our definitions (that we usually have pseudo hidden in a .inl file) to inside the class declaration.

We applied this strategy in several places reducing the bloatage by a considerable factor. For example, the vector library seems an ideal target because you normally only instantiate that library for a few types (float, double, int) and “big” functions like Invert() for 4×4 matrices can be hidden and exported from the dll. Another good candidate is std::string (instances for chat and wchar_t). Although, at least in MSVC, this job is already being done by the crt (not tested in other platforms).

I hope you find this useful. Thanks for reading and welcome to all the new subscribers since my last post!

Documentation is the part of the development process where usually less time and money is invested. Few resources implies a poor infrastructure for something that, as the project gets bigger and bigger, becomes a very important part of the overall process.

I want to share in this post the approach we are following for a project I have been involved with for over two years. This project (I will be able to give details about it very soon) is mainly a framework composed of libraries to be used by other companies. Basically, a work from developers to developers, clearly a context where documentation becomes very important.

The main objectives we tried to achieve with this architecture:

• Documentation must be located in the code repository. This means that each development branch has its own documentation that is integrated in the same way that the code.
• Documentation generation must be part of the building process. We use our build pipeline for everything: compile versions, test versions, compile data packages, generate installers, distribute symbols and, of course, build documentation. This means that the source of the documentation must be a very basic representation, easy to edit, that later is converted to a pdf, doc, html by the building process. This point implies that the documentation can be easily distributed to clients or even generated by themselves.
• Access to current documentation must be integrated into the development environment. In the same way I can review bugs in the bugtracker, stories in the task manager and browse the source code I must be able to read the current documentation from a web browser without having to rebuilt it manually.

We make the distinction between two kind of documentation: API Documentation and Technical Articles.

API documentation are those documents where class usage is described: description for each function, for each parameter, enumerations, etc. Doxygen seems the perfect fit for this purpose. But after several projects using it we opted for not using it in this project because we considered it a waste of time. Doxygen may be really useful to generate documentation for a library that is distributed without sources but one of the pillars of our development philosophy is to always provide source code to clients. This way, they can choose how to compile it: one dll, several dlls, one lib… and tweak the different configurations. Another good rule that we follow is that code must document itself. Doxygen would force us to write code like this:

class MemoryManager: public MemoryAllocator
{
    /// Allocates memory on a specified alignment boundary
    /// \param size Size in bytes of the block requested by the user
    /// \param alignment The alignment value, which must be an integer power of 2
    /// \return A pointer to the new allocated block of memory
    /// \remarks Thread-safe
    virtual void* AlignedAlloc(size_t size, size_t alignment) = 0;
}

Quite redundant code, don’t you think? Obviously AlignedAlloc() allocates memory that is aligned on a boundary, I do not need a comment for that! The same for the function parameters. So we ended up writing documentation for only the parts that are really confusing or need a clarification (for example, the fact that AlignedAlloc() is thread-safe) and using the source code as the documentation itself for both the interface and the implementation. We augment this documentation by using unit tests that serve as example usage for each class. We know this decision is a controversial one and may be in the future we return back to using Doxygen or something similar. But for now, it is working fine.

A technical article is like a manual that describes the architecture and philosophy of a specific part of the framework. They provide information to a deeper level than the mere description of classes provided by the API documentation. We usually link to technical articles from API documents.

Although a wiki seems an ideal candidate for writing technical articles we rejected it because it didn’t satisfy the points enumerated above. Years ago and inspired by the Gentoo Documentation Project I used DocBook for a similar purpose. I wrote about it in a previous article: Documentation using XML. And although the results were quite satisfactory I didn’t want to return to that solution, mainly because a XML format is hard to write, edit, maintain, etc. I think we found a better solution: reStructuredText. reStructuredText is part of the DocUtils suite and from very simple what-you-see-is-what-you-get plaintext markup it generates quite good looking documents. Another bonus feature that finally inclined the balance towards reStructuredText is that it perfectly integrates with Trac which we have been using since the beginning. Trac is a suite that integrates wiki, bugtracking, source code browsing and task management in a single web program. I like to see trac as a free alternative to FogBuz. Trac supports both its own wiki format and reStructuredText allowing us to integrate the documentation located in the repository as part of our wiki infrastructure satisfying the last point mention above. Time will say if this infrastructure will suffice.

Want to comment how do you solve the documentation issues in your project? Please do so. Comments are open. Thanks for reading.

Imagine, for example, a generic object factory where object instances are created using unique identifiers. The classical solution here is having a shared-by-all-code header where all the identifiers are declared in a C enumeration. This solution, apart from creating a serious physical dependency where adding a new identifier to the enumeration forces a recompilation for all the project, is unfeasible in modular architectures where modules are isolated. In those architectures having a global header is not an option.

One viable solution may be using strings as identifiers. But strings are heavy objects, hard to compare and prone to typing errors because miswritten symbols would be detected at run-time instead of compile-time. Other equally insufficient solutions to this problem include FourCC and esoteric template tricks for generating a hash at compile time (desist from it, it is not possible to solve this 100% with templates because strings cannot be used as template parameters and anyway hashing a string is not collision-free. More information in this usenet thread). Mick West proposes more solutions in his Practical Hash IDs article.

What follows is an implementation that has been working nicely for me and that satisfactorily fits the requirements for simulating compile-time strings. First let me show you two examples of the usage:

namespace
{
DECLARE_SYMBOL(CubeMesh)
DECLARE_SYMBOL(SphereMesh)
DECLARE_SYMBOL(DuckMesh)
}
 
void CollectNodes(Ptr<Node>& node)
{
    Ptr<Mesh> mesh0 = CreateObject(S(CubeMesh));
    node->Add(mesh0);
 
    Ptr<Mesh> mesh1 = CreateObject(S(SphereMesh));
    node->Add(mesh1);
 
    Ptr<Mesh> mesh2 = CreateObject(S(DuckMesh));
    node->Add(mesh2);
}

namespace
{
DECLARE_SYMBOL(FirstMessage)
DECLARE_SYMBOL(SecondMessage)
DECLARE_SYMBOL(ThirdMessage)
}
 
void ProcessMessage(const Message& msg)
{
    if (msg.id == S(FirstMessage))
    {
       /// ...
    }
    else if (msg.id == S(SecondMessage))
    {
       /// ...
    }
    else if (msg.id== S(ThirdMessage))
    {
       /// ...
    }
}

A symbol represents a compile-time string. They must be declared before being used. The macro for declaring a symbol is hiding an inline function with a static inside itself:

#define DECLARE_SYMBOL(id)\
    inline Symbol __GetSymbol##id() throw()\
    {\
        static size_t sym;\
        if (sym == 0)\
        {\
            sym = GetIdFromString(#id);\
        }\
        return Symbol(sym);\
    }

The function GetIdFromString() hashes the string, stores it in an internal table and returns the table position for that string (the Symbol class is a simple wrapper around the identifier). This is done only the first time the symbol is requested. For future requests the static ID is returned. This adds a little overhead against using simple integers as symbols. Beware of local static initializations: they are not thread-safe. That is the reason of the manual comparison against 0. GetIdFromString() must be thread-safe for this code to work.

The S macro simply invokes the local function previously generated:

#define S(id) __GetSymbol##id()

And there you have it. Compile-time strings with negligible (in case you are doing anything more that simply comparing symbols) overhead. In case you need 100% efficient code you could pre-generate a table with the symbols being used by your project (searching for all DECLARE_SYMBOL blocks) and substitute each S() with a really unique identifier generated at compile-time. And that would be so easy if the preprocessor could be extended in a standard way…

Hope this makes sense. Thank you for reading.

1. Practical Hash IDs
2. FourCC
3. Compile-time string hash generator

If you can do without that information the following code will help you to save a few bytes, even halving the size of the byte-code in the best cases.

Although not documented in the DirectX SDK, this CodeGem is not an undocumented hack. The Direct3D shader code format is documented in the MSDN, so it probaly won’t change in future revision of DirectX v9.0 (if there is going to be any more…)

D3DPtr<ID3DXBuffer> StripComments(const D3DPtr<ID3DXBuffer>& code)
{
    // Calculates the new size (without comments)
    int* codeData = static_cast<int*>(code->GetBufferPointer());
    unsigned int sizeInWords = code->GetBufferSize() / 4;
    unsigned int strippedSizeInWords = sizeInWords;
 
    for (unsigned int i = 0; i < sizeInWords; i++)
    {
        if ((codeData[i] & 0xffff) == D3DSIO_COMMENT)
        {
            int commentSize = codeData[i] >> 16;
            strippedSizeInWords -= 1 + commentSize;
            i += commentSize;
        }
    }
 
    // Creates a new buffer with the original code but omitting the comments
    D3DPtr<ID3DXBuffer> strippedCode;
    V(D3DXCreateBuffer(strippedSizeInWords * 4, strippedCode.GetPtrForInit()));
 
    int* strippedCodeData = static_cast<int*>(strippedCode->GetBufferPointer());
    size_t offset = 0;
 
    for (unsigned int i = 0; i < sizeInWords; i++)
    {
        if ((codeData[i] & 0xffff) == D3DSIO_COMMENT)
        {
            int commentSize = codeData[i] >> 16;
            i += commentSize;
        }
        else
        {
            strippedCodeData[offset++] = codeData[i];
        }
    }
 
    return strippedCode;
}

• Launchy. Launchy is a keystroke launcher that clones the behaviour of Quicksilver in Mac OS. With this tool you will say goodbye to your start menu and desktop icons. Everything is now accessible from a few keystrokes: folders, applications and even websites, all with a simple alt + space. The perfect complement for Launchy is StartKiller, a tool for removing the Start button from the taskbar. To me, Launchy is as revolutionary as 4DOS was back in MS-DOS days. How much time did you save? (bonus: and now that you can have a 100% clean desktop it is time to use a decent wallpaper…).
• AutoHotkey. Whatever can not be done with a few launchy keystrokes most likely can be programmed with an AutoHotkey macro. AutoHotkey incorporate a powerful script language that will allow you to automate almost anything: instant access to disk folders, internet tabs, activate tray programs, change visual studio layouts, send email, check calendar, etc.
• xplorer². xplorer² is a file manager with enough features to say good bye to Windows Explorer (Microsoft, admit it, it is not designed for advanced file manipulation). Although there is a professional version, the free lite version is enough to me, especially the Tabbed dual-pane interface feature. With AutoHotkey you can easily redirect Win + E to xplorer².

And that closes out my small contribution to reduce energy wastage in the world . I am sure you can share more gems like these. One more time, thanks for reading.

Hacker’s Delight
Author: Henry S. Warren, Jr.
Pages: 306
Published: 2003

You may think that I have become obsessed with books about hacking but this book is totally different from any of the others. In this book the term hacker is meant in the traditional sense (before the negative definition was popularized) of someone interested in understanding how things work and how to solve problems efficiently. Although the hacker term can be applied to whatever domain, in this book the domain is computing technology.

‘Hacker’s Delight’ is a book about bits and small programming tricks applied to machines. With ’small’ I mean that you won’t find here a description of the Merge sort or Radix sort but, for example, you will learn to determine in constant time if an integer is a power of two or not.

In more that 300 pages and with a mixture of pseudo assembler and C you will find all kind of tricks for arithmetic bounds, counting bits, searching bits, multiplications, elementary functions, floating point, etc. Even wondered if the base2 used by computers is the most efficient? This question and a lot more are covered in this book.

Although the book is a little bit oriented towards compiler developers, every ‘real’ programmer can get a huge benefit from reading and thoroughly understanding this book.

I read this book on several flights and really enjoyed this little gem book of tricks. I would definitely recommend to have it in your bookshelf.

Rating: 8 / 10

As you probably know I am working in a very small (3) team through internet. We do not share a physical place and we have very limited resources. All the infrastructure is based on servers we have at our own home (code repository, wiki, bug tracking service, build machines, web server, backup machines, NAS servers, etc). As you can imagine, we try to optimize our time and bandwidth as much as possible. I want to share with you in this post two examples of this optimizing philosophy with the idea of discussing them and discovering other interesting usages you may be doing (if you want to share of course)

• Twitter: I like to know where the rest of the team is working on. We have weekly voice meeting, we have emails and IM accounts but that is not enough to know with precision where each part is working on. Twitter, a micro-blogging service you probably know is ideal for this purpose. We have private twitter accounts (nobody out of the team can read it) where we update or current status: developing a new package, fixing a ticket, writing documentation, meeting a client, etc. With a simple look at your twitter account you get the status of the team.
• Dropbox: I am absolutely impressed with this software. If you don’t know about it I recommend that you have a look at its tutorial. The service is incredibly simple to use and it just works without problems. We have created a dropbox account for internal distribution and testing of our binary releases. Our build machine copy each generated distribution to a shared dropbox folder. This dropbox folder is shared with our team giving us the following advantages:
  • Every member on the team have the binaries everywhere and in all machines we want to test.
  • Logs for each execution are saved in the shared folder and are automatically synchronized in all the accounts. The logs give us useful information about the execution of the software that every developer can inspect.
  • Crashes are stored as minidumps in that same folder. And, as we distribute pdb with our releases, this means that everybody in the team can open any dump from any release and reproduce the exact crashing conditions everywhere. For more information about symbols, read my previous article about Setting up a Symbol Server

Do you have more interesting related ideas? Please, share them with us.

A good memory management architecture is one of those key features that can make the difference between a successful application and an unsuccessful one. For realtime applications the memory architecture becomes critical. This article discovers how memory management is more than tracking where your malloc() and free() are located. Although the focus will be on realtime applications implemented in C, all the techniques described here can be translated to other scenarios because the terms described are language-independent.

The best memory management is doing no allocation at all. You should architect your software to minimize the interaction with the memory manager. In the past that was a realistic option but in modern architectures that objective becomes harder to achieve. Modern requisites for realtime architectures like, for example, content streaming or hot loading force us to have frequent interactions with the memory manager. This article describes how this can be done efficiently. The ideas provided plus the links to other articles will give you enough information to implement your own solution. No downloadable code is provided with this article but if you need help implementing the ideas described here do not hesitate to contact the author.

The article is subdivided in three parts. Each part is dedicated to different memory management layers. From low-level to high-level, the first layer is the Memory Allocator. The Memory allocator, in direct contact with the operating system, implements the memory allocation/deallocation functionality. The next layer that will be described is dedicated to Pools, or how to minimize the interaction with the Memory Allocator. In the last part, automatic memory management or how to avoid human interaction with the memory manager is described.

1. The Memory Allocator

The Memory Allocator is a class implementation with basic memory management functionality. An interface for this class would be something like this:

////////////////////////////////////////////////////////////////////////////
/// MemoryAllocator. Base class for all memory allocators
////////////////////////////////////////////////////////////////////////////
class MemoryAllocator
{
public:
    virtual ~MemoryAllocator() = 0 {};
 
    /// \return The name of the allocator implementing this interface
    virtual const char* GetName() const = 0;
 
    virtual void* Alloc(size_t size) = 0;
    virtual void* Realloc(void* ptr, size_t size) = 0;
    virtual void Dealloc(void* ptr) = 0;
 
    /// Returns the size of a memory block allocated by this allocator
    virtual size_t SizeOf(void* ptr) const = 0;
 
    /// Log statistics
    virtual void DumpStats() const = 0;
};

This proposed interface is virtual and a concrete implementation must be provided. Probably you will implement this removing the virtual class, but having it is a good option to reduce compile-time dependencies. Anyway the virtual functions will not be a bottleneck.

The first implementation for this interface is obvious: implement it using the standard C functions like malloc, realloc and free. The SizeOf() function can be implemented in Visual Studio with _msize. Rest of the platforms provide similar functionality. This first implementation may be enough for you. Test it, profile it and if you need better performance pass to the next step.

The next step starts with a piece of advice: do not implement your own allocator. Do not reinvent the wheel. Do not waste your precious time implementing something that probably others have already done.

What follows is a description of desirable properties that must have a good memory allocator. Next, links to existing implementations are provided.

Properties of a good memory allocator

When looking for existing memory allocator implementations you must pay attention at the following features:

• Efficient for small block allocations: a high percentage of your allocations are for small blocks (from a byte to a few hundred). The allocation for this blocks must be very fast and the size overhead must be minimum. When you request N bytes to an allocator, it is normal that internally more than N bytes are allocated due to internal headers. A good small allocator must allocate/deallocate in O(1) time and add zero overhead. Think about it, for a small block a common overhead of 8 bytes can be quite a lot! Is it possible to have zero-bytes overhead? Of course, separating your small allocators from the big one and playing with virtual addresses that objective is achievable
• Efficient for multithreaded programs: unlike in the past, multithreading is actually not an option for your architecture, it is mandatory. A good memory allocator must provide smarter mechanisms than protecting each function with a lock. Using Thread-Local-Storage (TLS) the locking mechanism can be avoided.

Although the implementation provided by Visual Studio have been improved a lot (last version tested: Visual Studio 2005) it is not especially good at the points described before. Small blocks have a high overhead and the functions are protected by thread locks, the worst way to implement a thread-safe memory allocator.

Known good implementations like Doug Lea Malloc Allocator [1] are not so good in multithreaded environments. From the author himself (about thread safety): “This is not especially fast, and can be a major bottleneck. If you are using malloc in a concurrent program, consider instead using ptmalloc, which is derived from a version of this malloc. (See http://www.malloc.de).”

The Hoard Memory Allocator [3] is very good at the two points described before. It is specially designed to be very fast on multiprocessor machines. It is licensed under the GNU terms. If you want to use it in a commercial application they offer commercial licenses in its website.

Inside the Google Performance Tools [4] there is a very efficient implementation of malloc. As described by the authors: “The fastest malloc we’ve seen”. Apart from the marketing issues, their implementation satisfies the two requisites described before.

You must test each implementation in your scenario and choose the one that better suits you. Architect the memory manager in a way that the internal allocator can be easily swapped. Your first implementation should be based on the standard malloc.

The allocator implementations described here have been tested in real and finished applications. There is an important point to emphasize here. Apart from the efficiency of your final applications you want to improve the day to day job when you are building your program. In those cases, efficiency is not as important as robustness. The Debug Implementation of Malloc() provided by Microsoft in their Microsoft Visual Studio is incredibly good at this point, detecting all kind of errors related to memory. This can save a lot of valuable time. So, one more time, being able to swap memory allocators is something you definitely want for your architecture.

Redirecting to your allocator

Having chosen an allocator the next step is redirecting all the program allocations through it. The first and most obvious solution is doing it explicitly, like the following example.

// Allocate a block from the memory manager
void *data = MemoryManager::Alloc(256);                                    
 
// Use the block
// ...
 
// Free the block
MemoryManager::DeAlloc(ptr);

And for C++ classes, the global operators new and delete must be overridden.

////////////////////////////////////////////////////////////////////////////
void* operator new(size_t size)
{
    return MemoryManager::Alloc(size);
}
 
////////////////////////////////////////////////////////////////////////////
void* operator new[](size_t size)
{
    return MemoryManager::Alloc(size);
}
////////////////////////////////////////////////////////////////////////////
void operator delete(void* ptr)
{
    MemoryManager::DeAlloc(ptr);
}
 
////////////////////////////////////////////////////////////////////////////
void operator delete[](void* ptr)
{
    MemoryManager::DeAlloc(ptr);
}

Apart from being insufficient, because there will be allocations that will not be caught with this method, it is prone to error. Users of this architecture have to remember that instead of using the malloc() functions they have to use other methods. Apart for creating less portable code, there is a great risk of losing allocations by not following that rule. Furthermore, code that can not be modified will not be redirected to your allocator. This includes all kind of Third-Parties you may be using. For example, DirectX or OpenGL allocations will not be caught. And last but not least this method is incompatible with other Third-parties doing the same thing (like overriding the operator new). You will be faced with hard to solve linking problems here. An example of Third-Party overriding the operator new and delete is the MFC.

To overcome all these problems platform specific solutions are needed. Some platforms (in many consoles for example) offer clean mechanisms to catch all the memory allocations. In other more heterogeneous platforms, like Windows PC, things are a little bit more complex. A method very similar to the one in [5] is described here for the Windows platform.

For catching all the memory allocations, the relevant memory functions will be ‘hooked’. The mechanism of hooking a function consists in inserting an assembler jump instruction at the beginning of the function. This way, all the code invoking the original function will be redirect to the new implementation. In addition, the original function code must be preserved because it will be needed. A copy of the beginning of the function, the prologue, is stored before patching it.

Commercial libraries like Detours [6] offer the hooking mechanism described. Free alternatives like the TRET toolkit [7] or the google-perftools [4] offer the same functionality. But if you do not want to incorporate such big packages into your project you can implement your own hooking mechanism. You are going to need a disassembler to properly implement it. Libdasm [8] is a recommended package for this purpose.

Having the hooking mechanism ready, now the target functions to be hooked must be selected. Standard C functions like malloc(), calloc(), realloc() and free() functions must be hooked. Global operators new and delete must be hooked too. But that is not enough, because under Windows lots of allocations come directly from functions like HeapAlloc(). At least this function family must be hooked too. [9] is a good lecture to discover how the virtual process address is populated in a Windows platform.

////////////////////////////////////////////////////////////////////////////
// Copy prologues for original CRT memory functions
OrgMalloc = static_cast<MallocT>(CopyPrologue(&malloc));
OrgCalloc = static_cast<CallocT>(CopyPrologue(&calloc));
OrgRealloc = static_cast<ReallocT>(CopyPrologue(&realloc));
OrgFree = static_cast<FreeT>(CopyPrologue(&free));
OrgMSize = static_cast<MSizeT>(CopyPrologue(&_msize));
 
// Hook CRT memory functions    
mCrtPatches[0] = PatchFunction(&malloc, &NsAlloc);
mCrtPatches[1] = PatchFunction(&calloc, &NsCalloc);
mCrtPatches[2] = PatchFunction(&realloc, &NsRealloc);
mCrtPatches[3] = PatchFunction(&free, &NsDealloc);
mCrtPatches[4] = PatchFunction(&_msize, &NsMSize);
 
// Obtain the CRT HMODULE because we need to patch operatow new/delete 
// functions and we cannot get a pointer to them
HMODULE crtModule;
GetModuleHandleEx(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS | 
    GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT, 
    reinterpret_cast<LPCTSTR>(&malloc), &crtModule);
 
// operator new        
mCrtPatches[5] = PatchFunction(crtModule, "??2@YAPAXI@Z", &NsAlloc);
// operator delete
 
 
mCrtPatches[6] = PatchFunction(crtModule, "??3@YAXPAX@Z", &NsDealloc);
// operator new[]
mCrtPatches[7] = PatchFunction(crtModule, "??_U@YAPAXI@Z", &NsAlloc);
// operator delete[]
mCrtPatches[8] = PatchFunction(crtModule, "??_V@YAXPAX@Z", &NsDealloc);
 
// Copy prologues for Windows memory functions    
OrgHeapAlloc = static_cast<HeapAllocT>(CopyPrologue(&HeapAlloc));
OrgHeapRealloc = static_cast<HeapReallocT>(CopyPrologue(&HeapReAlloc));
OrgHeapFree = static_cast<HeapFreeT>(CopyPrologue(&HeapFree));
 
// Hook Windows Heap functions
HMODULE hModule = GetModuleHandle(NST("kernel32"));
mW32Patches[0] = PatchFunction(hModule, "HeapAlloc", &NsHeapAlloc);
mW32Patches[1] = PatchFunction(hModule, "HeapReAlloc", &NsHeapRealloc);
mW32Patches[2] = PatchFunction(hModule, "HeapFree", &NsHeapFree);

Care must be taken to avoid infinite recursions. For example, the CRT memory functions are implemented using Windows Heap functions. This can be a problem if your memory allocator is implemented with malloc and free. Another source of problems can be found when you have several versions of the CRT in the same process. This happens when distinct dynamic libraries are linked with the static library version of the CRT.

The recommended settings are as follows. First, link as much libraries as possible with the DLL version of the CRT. This is the CRT that will be used by the memory manager too. Second, hook the malloc functions from this DLL to be redirected to your allocator. And third, hook the HeapAlloc functions to redirect only the requests being done in heaps distinct from the CRT used by the memory manager. Requests coming from the main CRT will be hooked by the malloc functions.

This is illustrated with the following example:

////////////////////////////////////////////////////////////////////////////
LPVOID WINAPI NsHeapAlloc(HANDLE hHeap, DWORD dwFlags, 
    DWORD_PTR dwBytes) 
{        
    void* ptr = OrgHeapAlloc(hHeap, dwFlags, dwBytes);
 
    if (hHeap != reinterpret_cast<HANDLE>(_get_heap_handle()))
    {
        MemoryManager::Track(ptr, dwBytes);
    }    
 
    return ptr;       
}

Memory tracking

Apart from the redirection mechanism to your own allocator, having control of all the memory allocations allows you to implement different tracking mechanisms that can give you valuable information. The tracking mechanism should be only active in the selected build configurations because it can add a considerable overhead. Two examples of tracking mechanisms are briefly described:

• Leak detector: the memory manager can keep a list of the currently allocated blocks with extra information about them: when it was allocated, which thread requested it, in which module, stack trace, etc. When this tracking mechanism is active, the memory manager increases the memory allocation sizes by 4 bytes (a pointer, 8 bytes in 64 bits) to hold a pointer to this node info. When your program exits, the memory manager can report a list of the blocks still allocated and that are, probably, leaks. Having a leak detector active while you are building your software is an invaluable feature that will improve the quality of your architecture.
• Memory Stats: with a tagging mechanism you can have exact control of where your memory allocations are being requested. Among all the tagging mechanisms, the stack based is the less intrusive and probably more flexible. Whenever a new memory block is requested to the memory manager, it is attached to the current stat block.

  The current stat block can be pushed / popped with macros like in the following example:

  void DX9RenderSystem::Init()
  {  
      NS_PROFILE("rendersystem", Mem);    
   
      mD3D = Direct3DCreate9(D3D_SDK_VERSION);    
   
  ...

  In the above example, all the memory allocated by the Init() function (and by all the functions called from it, including SLT allocations, etc) are accumulated in the “rendersystem” node. When exiting that function, the “rendersystem” node is popped from the stack.

  Each frame, the memory manager could collect all the node stats and information about the memory usage could be listed. For example:

  Node                            Memory         Memory (including children)
  --------------------------------------------------------------------------
  *root                  0%      10667320 ( 56%)    19029395 (100%)
  RenderSystem          56%       6381409 ( 34%)     6386525 ( 34%)
  AudioSystem           90%       1835510 ( 10%)     1836646 ( 10%)
  reflection            99%        105028 (  1%)      106576 (  1%)
  symbols              100%         29300 (  0%)       29300 (  0%)
  factory              100%         10800 (  0%)       11352 (  0%)
  --------------------------------------------------------------------------

2. Memory Pools

A memory pool preallocates a number of fixed size memory blocks from the memory allocator. The pool offers the same functionality than the memory allocator but due to the fixed size restriction it can offer several important advantages:

• Avoiding Dynamic Memory Allocation: except for preallocated blocks, whose memory is requested to the memory allocator when the pools are initialized, each allocation operation done by the Memory Pool implies no heap allocation. This results in faster operation times, nearly in the same order of magnitude than stack based allocations
• Constant time for Allocation / Deallocation: thanks to the fixed size restriction, an optimal implementation of a Memory Pool can allocate and deallocate block in O(1) time
• Cache coherence: all the blocks preallocated in the Memory Pool are located in a contiguous block of memory improving the cache performance
• Minimum Memory Fragmentation: in a Memory Pool there is no external fragmentation and the internal memory fragmentation when the pool is full is insignificant (there is only internal memory fragmentation in the big block allocated at initialization time)

Small objects (few bytes) that cannot be stored in the stack should me managed by Pools. A good implementation can be found in Alexandrescu’s book [10]. To encapsulate the pool usage class operator new and delete can be defined in the following way:

////////////////////////////////////////////////////////////////////////////
#define MANAGED_BY_POOL(Pool)\
    static void *operator new(NsSize size)\
    {\      
        return Pool->Allocate();\
    }\
\
    static void operator delete(void* ptr)\
    {\
        Pool->Deallocate(ptr);\
    }\
\
    static void *operator new(NsSize size, void* placementPtr)\
    {\       
        return placementPtr;\
    }\
\
    static void operator delete(void* ptr, void* placementPtr)\
    {\
        NS_UNUSED(ptr);\
        NS_UNUSED(placementPtr);\
    }

The following example implements a small struct that is transparently managed by a pool.

extern ObjectPool<Point, 32>* gPointPool;
 
////////////////////////////////////////////////////////////////////////////
struct Point
{
    NsFloat32 x, y, z;
 
    MANAGED_BY_POOL(gPointPool);
};

STL pools

STL is the perfect place where to use Memory Pools. Node base containers (list, map, hash_map) make heavy use of dynamic memory. To avoid the costs described above it is recommended to use Pools with STL. Memory used by STL containers is controlled by allocators that are passed to the container when it is created. [11] is a very recommended read that gives a good description of how to implement STL allocators. There you will learn how to code an allocator that uses a memory pool.

struct TrackInfo
{
    NsSize size;
    MemProfilerNode* profilerNode;
};
 
typedef std::map<
    const void*, 
    TrackInfo, 
    std::less<const void*>, 
    PoolStlAllocator<std::pair<const void*, TrackInfo>, 1024> 
 
> ExternalTrack;
 
////////////////////////////////////////////////////////////////////////////
ExternalTrack mExternalTrack;

3. Garbage Collection and Smart Pointers

The less interaction you have with the memory manager the better. Memory management errors are probably one of the worst problems in software engineering. Memory leaks, buffer overruns, etc. That is why modern languages like Java, Python and C# try to avoid exposing memory management to programmers using memory garbage collection techniques.

In C++ there is no automatic garbage collection but it can be implemented using object reference counters and smart pointers. This way, manual new and deletes can be avoided reducing the probability of incurring into memory errors. In Alexandrescu’s book [12] all the details for implementing smart pointers can be found.

To implement the reference counter mechanism each class must derive from a base class defining an integer for the counter and public functions AddReference() and Release() to manage the counter. The Release() function is in charge of deleting the instance when its counter reaches to zero. An intrusive implementation for reference counting where the counter is stored in the class itself is the recommended one for reasons of efficiency.

////////////////////////////////////////////////////////////////////////////
class BaseRefCounted: public BaseObject
{
public:
    /// Default constructor.
    BaseRefCounted();
 
    /// Increments reference count
    /// \returns Number of references after incrementing one
    int AddReference() const;
 
    /// Decrements reference count, deleting the object when reaches 0
    /// \returns Number of references after releasing one
    int Release() const;
 
    /// Gets current reference count for the object
    /// \return Object number of references
    int GetNumReferences() const;
 
protected:
    /// Destructor. Base classes are abstract classes. Destructor is pure 
    /// virtual. This destructor is declared protected to avoid deleting 
    /// reference counted objects. Release() should be used in this case.
    virtual ~BaseRefCounted() = 0;
 
private:
    /// Base classes are non-copyable objects
    //@{
    BaseRefCounted(const BaseRefCounted&);
    BaseRefCounted& operator=(const BaseRefCounted&);
    //@}
 
    volatile mutable int mRefCount;
};

There are alternatives to reference counting (read [13] for a mark-sweep algorithm) but it is probably not worth the effort using such a sophisticated algorithm if you are mixing in your program C++ code with other high level scripting language where garbage collection is implemented natively.

The code becomes a lot cleaner and safer when using smart pointers:

////////////////////////////////////////////////////////////////////////////
/// d0 and d1 are automatically destroyed when no code references them
////////////////////////////////////////////////////////////////////////////
Ptr<IStream> d0 = fileSystem->OpenRead("Dat/Stats/s0.dat");
d0->Read(buff, 512);
 
Ptr<IStream> d1 = fileSystem->OpenRead("Dat/Stats/s0.dat");
d1->Read(buff, 512);

4. Conclusion

This article described three memory management layers: the memory allocator, the pool allocator and the garbage collector allocator. Although memory management is important in the architecture, its usage must be as transparent as possible. This is the principle behind the layer subdivision proposed in this article.

Thanks for reading. Comments are opened for discussing anything related to it: bugs, suggestions, improvements, questions…
 

1. Doug Lea Memory Allocator (dlmalloc)
2. Wolfram Gloger’s malloc homepage
3. The Hoard Memory Allocator
4. google-perftools. Fast, multi-threaded malloc() and nifty performance analysis tools
5. Monitoring Your PC’s Memory Usage For Game Development
6. Detours instrumenting library homepage
7. TRET: The Reverse Engineering Toolkit
8. libdasm: simple x86 disassembly library
9. Why Your Windows Game Won’t Run In 2,147,352,576 Bytes
10. Loki Library – Small Object Implementation
11. Pete Isensee on Custom STL Allocators
12. Modern C++ Design
13. A garbage collector for C and C++

Oviedo will hold the fourth edition of the GameLab conferences on July 10th – 11th. Undoubtedly the place to be if you want to meet lot of interesting people related to the Videogames industry. This event is growing bigger and bigger each time and is becoming a point of reference here in Spain.

Like the last year I will be giving a course on Advanced real-time 3D techniques the first day.

Hope to see you there!

This weekend just finished the course I have been giving at the Oviedo University. The course is about programming graphic engines for videogames. In 30 hours / 6 days I tried to explain how to architect a solid engine for realtime purposes.

This is the first time I talk about the task I have been involved in the last months: Noesis Engine (a provisional name). Till now, it has been developed by a very small team and contributed to two commercial products. A small videogame is under construction now. I expect to give more information about this in the future.

A link to the first session of the course: Noesis – Core. The document reveals not too much information if you are not attending the class, but may be you find something interesting there (or wrong, and we can discuss). The first part is a global introduction to the course, the second one is about the core technology being used for the rest of the course. The document is in Spanish, I have no time now to translate it (I would be really grateful to any volunteer helping in this). Sorry for that.

And following with Spanish documents, I contributed to several tutorials in codepixel, a daily mandatory read if you understand Spanish, about the same topic, Graphics in Realtime. The hard part was done by Javier Loureiro / derethor. Iq / RGBA helped to this documents too, The tutorials:

• Introduction
• The Graphic Driver
• The Scene Graph

And nothing more for today. As you can see I am still alive and working really hard.

UPDATE: Thanks to Ricardo Amores and Miguel Herrero for translating the powerpoint to English. It can be downloaded from: Noesis – Core – Eng