Do you want to stay ahead of the curve and start to use polymorphic allocators in your projects? Are you undeterred even by the cost of virtual calls? If so, it's time to talk about the nuances with a lifetime and why you can't just replace containers with analogs from the 'pmr' namespace.
Now, imagine, just as an experiment, that you're working in a biological lab. Your supervisors have tasked you with developing an app to simulate the bacterial life cycle. So, you design a class that represents a bacterium. In this case, each bacterium contains a set of genes. It could look like this:
class Bacteria
{
private:
using gene_type = TheGene;
using genes_container = std::vector<gene_type>;
private:
static genes_container RandomGenes();
public:
Bacteria() = default;
Bacteria(const Bacteria&) = default;
Bacteria(Bacteria&&) = default;
Bacteria& operator = (const Bacteria&) = default;
Bacteria& operator = (Bacteria&&) = default;
~Bacteria() = default;
public:
void MutateRandomGene();
Bacteria Clone() const;
/* something else ... */
private:
genes_container m_genes = RandomGenes();
};Your app works, bacteria are simulated—but you notice a slowdown. In the process of profiling your code, you stopped at the Bacteria class. After some investigations, you look at this pinnacle of IT art and suspect it may be the bottleneck. But why?
A lot of bacteria are regularly dividing and dying. Thousands and millions of them. Each bacterium has its own container with its genes. Whenever you create the bacterium with the set of genes, std::vector accesses std::allocator to allocate memory for the genes. The standard allocator "knocks" on the system using the operator new, begging for a chunk of available memory. That's slow—much slower than accessing memory already owned by the process.
Moreover, when the bacterium dies, the hard-won memory returns to the system because the standard allocator uses the operator delete. Meh, you could reuse that memory to store the data of a new bacterium. Instead, the standard allocator will go cap in hand to the system again, pleading for another chunk of RAM...
This is a well-known issue, and devs have created many custom allocators long ago to address it. However, keep in mind that you should manually implement them or use the third-party libraries—that's not always the best option. Oh, how it'd be nice if the language had an out-of-the-box support for something like this...
The developers' prayers have been answered! Starting with C++17, you can use polymorphic allocators. Their idea is a little different from what we need. What is important is that they allow you to specify a buffer from which the allocator will take memory. Such a buffer is called a resource, and it must be inherited from std::memory_resource. It's also important that C++17 introduced several standard resources. Let's examine std::unsynchronized_pool_resource as an example. I should admit that all options mentioned in this article mostly apply to all standard library resources.
In std::unsynchronized_pool_resource, if the allocation size doesn't exceed the max size, memory is released and returned to the system only when the resource's destructor is called. Until then, the memory released by the container returns to the pool, not to the system. It means that the process owns all memory as long as the pool is alive. Consequently, you can reuse it without accessing the system.
You think, "Perfect! Exactly what I need!" You eagerly change the gene container to std::pmr::vector. There is no issue with this since you've aliased your classes beforehand. Now, you also need to find a place where you can store the shared pool for your bacteria. You add the static function that returns your pool. Now the class looks like this:
class Bacteria
{
private:
using gene_type = TheGene;
using genes_container = std::pmr::vector<gene_type>;
using pool_type = std::pmr::unsynchronized_pool_resource;
private:
static genes_container RandomGenes();
static pool_type& GetPool();
public:
Bacteria() = default;
Bacteria(const Bacteria&) = default;
Bacteria(Bacteria&&) = default;
Bacteria& operator = (const Bacteria&) = default;
Bacteria& operator = (Bacteria&&) = default;
~Bacteria() = default;
public:
void MutateRandomGene();
Bacteria Clone() const;
/* something else ... */
private:
genes_container m_genes = RandomGenes();
};Licking your lips, you cast the benchmarks, run them, and... The results are unimpressive, meh.
Here comes the first nuance of working with polymorphic allocators. When using polymorphic allocators, the behavior of constructors and copy/move operators can differ from what we are used to.
When the copy vector constructor is called, the object being constructed gets the allocator via a call to the select_on_container_copy_construction function. The standard std::allocator doesn't have such a function, so the allocator of the constructed object is equal to the allocator of the copied object. But std::pmr::polymorphic_allocator does have such a function. As a result, instead of taking the allocator from the copied object, the constructed object gets the default allocator. If you haven't changed it via the std::pmr::set_default_resource function, it'll use the operators new and delete in the same way instead of accessing the shared memory pool. Yeah, you hit the brakes again...
No worries, though. It's necessary that the copied bacterium uses the same memory pool as the original bacterium. Let's explicitly define the copy constructor and specify that an allocator should be used for the recipient:
Bacteria::Bacteria(const Bacteria &other)
: m_genes { other.m_genes, other.m_genes.get_allocator() }
{}Fortunately, there's no such issue with the move constructor as std::vector will take the allocator directly from the source object. I'm not sure why the developers chose to handle it this way, but it certainly seems non-obvious and inconvenient. If you have any idea why, feel free to share it in the comments.
Anyway, you go back to benchmarks and... It looks much better. But it wouldn't be fun if that were the end of it, right? When using the polymorphic allocator, you must ensure that the resource's destructor isn't called before the last object's destructor that uses it.
For example, imagine you write a class to describe an incubator for bacteria:
template <typename B>
class Incubator
{
private:
using bacteria_type = B;
using bacteries_container = std::deque<bacteria_type>;
private:
Incubator() = default;
Incubator(const Incubator&) = default;
Incubator(Incubator&&) = default;
Incubator& operator = (const Incubator&) = default;
Incubator& operator = (Incubator&&) = default;
public:
~Incubator();
public:
void AddBacteria(bacteria_type bacteria);
/* something useful */
public:
bacteries_container bacteries;
};Let it be static too:
Incubator<Bacteria>& GetBacteriaIncubator()
{
static Incubator<Bacteria> incubator;
return incubator;
}The pitfall here is subtle, but it's something an unwary user might easily overlook. A developer that uses your library/classes will probably write something like this:
auto &&incubator = GetBacteriaIncubator();
incubator.AddBacteria({});And will get undefined behavior. This may come as a nasty surprise for anyone unaware of the implementation details of the Bacteria class. We're used to the fact that with the standard allocator, we don't have to care much about variable lifetime. Everything is straightforward with memory: allocate when created and release when destroyed. However, polymorphic allocators demand extra attention because the object lifetime doesn't exceed the memory_resource lifetime.
In the example above, the Incubator class would be initialized first, followed by the bacteria pool. Since both have static storage duration, they'll be destroyed in the reverse order to creation. The pool destructor will be called before the incubator destructor. As a result, if there are bacteria in the incubator when they try to release memory, they'll turn to the polymorphic allocator, which will have a "dangling" pointer to the pool.
Let's say that we don't want to use the static pool now—give bacteria power to its own resource! While just one bacterium is alive, the pool lives too. Then, when all the bacteria die, the pool dies consequently. At the same time, we'll release memory instead of keeping unused memory until the program exits.
Let's try adding the std::shared_ptr data member to bacteria that references the resource for the allocator. They'll own one shared pool that will be destroyed when all bacteria die. Well, I suppose that an experienced developer might already sense trouble brewing... Let's not jump ahead. For now, here's how the bacterium looks:
class Bacteria
{
private:
using gene_type = TheGene;
using genes_container = std::pmr::vector<gene_type>;
using pool_type = std::pmr::unsynchronized_pool_resource;
private:
static genes_container RandomGenes(pool_type &pool);
public:
Bacteria();
Bacteria(const Bacteria&);
Bacteria(Bacteria&&) = default;
Bacteria& operator = (const Bacteria&) = default;
Bacteria& operator = (Bacteria&&) = default;
~Bacteria() = default;
public:
void MutateRandomGene();
Bacteria Clone() const;
/* something else ... */
private:
std::shared_ptr<pool_type> m_pool;
genes_container m_genes;
};Don't forget about constructors:
Bacteria::Bacteria()
: m_pool(std::make_shared<pool_type>())
, m_genes(RandomGenes(*m_pool))
{}
Bacteria::Bacteria(const Bacteria &other)
: m_pool(other.m_pool)
, m_genes(other.m_genes, other.m_genes.get_allocator())
{}Well, everything seems to be working: allocations are fast, memory is released when needed—what else you need? Now let's try to create not only bacterial multiplication but also new pathogens:
if (RandomDie() && bacteries.size() > 1)
{
bacteries.erase(bacteries.begin() + index);
}
else if (RandomDivision())
{
auto &&bacteria = bacteries[index];
bacteries.push_back(bacteria.Clone());
}
else if (RandomNewBacteria())
{
bacteries.emplace_back();
}Boom! Combo! You hit a crash and undefined behavior. In the implementation, a new bacterium—one that isn't copied from another—will use a fresh memory pool instead of the shared one. As a result, two random bacteria may have m_pool pointers referencing different pools. On its own, this doesn't cause any errors, however, we're again inefficiently using memory. The program crashes due to the dangling pointer to the pool.
But how can that be? After all, you use the smart pointer, which should ensure that the object is alive. The snag stems from the copy/move operators that are called when an element is deleted. After deletion, the vector attempts to shift the remaining elements to the left. We've explicitly specified that the compiler should generate the copy/move operators. Next is the implementation-defined behavior, but it leads to something like this:
Bacteria& Bacteria::operator = (const Bacteria &other)
{
if (this == &other) { return *this; }
m_pool = other.m_pool;
m_genes = other.m_genes;
return *this;
}In just a few lines, there are two distinct causes of UB. First, std::shared_ptr is copied, and then the container that uses it. If m_pool is a single strong reference, the resource will be deleted when the copy statement is called. Next, when std::vector is copied with the polymorphic allocator in the = operator, the allocators will be compared via the == operator. In turn, the polymorphic allocator will return the result of the underlying resource comparison by dereferencing the pointer to the pool that's already dead. It means that the dangling pointer will be dereferenced. The same dangerous scenario applies to the move operation as well.
The default behavior of the copy/move operators doesn't suit our needs. Let's implement our version where the pool can't be deleted while the container still uses it:
Bacteria& Bacteria::operator = (const Bacteria &other)
{
if (this == &other) { return *this; }
m_genes = other.m_genes;
m_pool = other.m_pool;
return *this;
}As I said before, there will be a second UB case. At the beginning of the article, I highlighted that the behavior of copy/move operators in containers with the polymorphic allocator deviates from what you might typically expect.
In this case, the polymorphic allocator isn't copied from other.m_genes when copying the container—it remains unchanged. That is, even after the copy operator is called, the old allocator with the pointer to the old pool will remain in m_genes. This means that even after calling the copy operator, the allocator in m_genes will still point to the old memory pool. You can avoid it only via explicit passing a new allocator.
However, in a typical assignment, you can't explicitly specify the allocator to be used. This leaves you with only one practical option: using a constructor.
Bacteria& Bacteria::operator = (const Bacteria &other)
{
if (this == &other) { return *this; }
m_genes.~genes_container();
new (&m_genes) genes_container(other.m_genes,
other.m_genes.get_allocator());
m_pool = other.m_pool;
return *this;
}However, it's difficult here either. After calling the destructor, we call the copy constructor, which may throw an exception. If that happens, the exception will be passed further, and during stack unwinding, the destructor can be called for *this. This will lead to a repeated call of the m_genes destructor. Here's UB again. Even if you wrap the code in the try construct when the exception is thrown, m_genes will be uninitialized and its destructor will be called again. Let's cheat a little bit!
Bacteria& Bacteria::operator = (const Bacteria &other)
{
if (this == &other) { return *this; }
auto copy = other;
this->~Bacteria();
new (this) Bacteria(std::move(copy));
return *this;
}Usually, if you use the polymorphic allocator, your constructor and move operator lose noexcept. This happens because allocators can point to various resources, which can potentially lead to memory allocations (for example, via the operator new). Since memory allocations can fail, exceptions like std::bad_alloc may occur.
To make the magic work, you also need to define the move constructor in the Bacterium class:
Bacteria::Bacteria(Bacteria &&other) noexcept
: m_pool(std::move(other.m_pool))
, m_genes(std::move(other.m_genes), other.m_genes.get_allocator())
{}Now, m_genes uses the same allocator as other.m_genes. So, no memory allocations will occur, and we won't get a suddenly thrown exception.
What a mess, right? But at least it works now.
Writing code like the examples above is a laborious endeavor. When using polymorphic allocators, however, if the application claims to be at all functional, you must ensure that resources have sufficient lifetimes. This is the main difference between polymorphic allocators and default allocators.
In my opinion, there is no 100% safe solution, but there are still steps that you can take. Let's introduce a resource manager—an entity responsible for managing your resources. You should create it at the very start of the program and ensure that all resources are obtained only from it. This way, all resources will be in one place, and the chance of objects leaking to some place with a longer lifetime tends to be zero.
The simplest implementation might look like this:
class ResourceManager
{
private:
using genes_resource_type = std::pmr::unsynchronized_pool_resource;
using other_resource_type = ....;
public:
static void Init();
static ResourceManager& Get();
private:
ResourceManager() = default;
public:
ResourceManager(const ResourceManager&) = delete;
ResourceManager(ResourceManager&&) = delete;
ResourceManager& operator = (const ResourceManager&) = delete;
ResourceManager& operator = (ResourceManager&&) = delete;
~ResourceManager() = default;
public:
genes_resource_type& GetGenesResource();
other_resource_type& GetOtherResource();
private:
genes_resource_type m_genesResource;
other_resource_type m_otherResource;
/* and etc. */
};If you initialize such a manager at the start of the program and obtain resources only from it, everything will probably be fine. Yes, you'll have to add an entry at the beginning of the main function, but you'll be less likely to catch UB. Applied to our previous examples, it'd look like this.
Yeah, you can't completely protect yourself. Someone can go into the project and write some code before initializing resource manager. It looks like this. But there's nothing you can do.
The manager implementation shown above is the simplest approach. If you want, you can (should) enhance the resource manager: implement automated memory release, enable dynamic allocation of new pools, and allow retrieval by index or tag. You can think of a lot of things.
However, the key takeaway from this article is that you can't just replace your standard allocators with polymorphic ones. Use them wisely and keep in mind possible surprises like a lifetime.