Webinar: Evaluation - 05.12
Or why I think developers need to know about them but should not use them.
Do you remember the first time you put a pointer to the first array element to the sizeof operator, and your code stopped working the way you intended? It certainly doesn't come close to the thrill of sticking your fingers in a power socket but...
Here's an array:
int arr[5] = {1, 2, 3, 4, 5};
And here it turned into a pointer:
int *ptr = arr;
The array-to-pointer conversion happened, we lost information about the array size and some nerve on top of that. Don't get me wrong: before using an array, one should first learn how to "cook" it, just like any other feature of our favorite language. But I don't like it when language rules seem to be constantly trying to trick the programmer and make things more complicated for no reason.
I think the problem is that this infamous array-to-pointer conversion imposes a certain mindset. It assumes that an array and a pointer to its first element are absolute equivalents and always lead to the same behavior when the same code is used. However, arrays and pointers are far from being interchangeable. Moreover, C++ has contexts where you may regret using one instead of the other.
First, let's get some very basic things straight from the start. A pointer has its own type, namely the compound type of the pointer-to-T. An array is a separate entity, so it has a different type than a pointer. Why don't we check it right off the bat?
To determine the type, let's use the typeid built-in operator and the name() function that returns a string containing the type name. The typeid operator works over RTTI, so the string differs depending on the compiler. However, the type is still determined as the same one.
Yes, typeid discards references and cv-qualification. We don't apply these concepts specifically in this part of the article. If we had used typeid to find out how a type is deducted in templates, we'd be stepping on a massive rake. Fortunately, there are no such rakes in our garden, we're just looking at the type of the explicitly declared variable.
We declare an array and look at its type:
int arr[3] = { 1, 2, 3 };
std::cout << typeid(arr).name() << std::endl;
Our compiler defines it as A3_i, an array of three elements of the int type.
At the same time, we can use the typeid operator on a pointer to the first element of an array:
int *ptr = arr;
std::cout << typeid(ptr).name() << std::endl;
In this case, Pi, which is a pointer to int, is printed to stdout.
If the types are different, where does the confusion come from?
The confusion in using arrays and pointers comes from the fact that C++, as well as its direct ancestor C, has contexts where an array becomes a pointer. Technically, it's hard to find contexts where an array is used by itself! Let's leave the reasons for these sudden changes out of this article and just say thanks to good old Dennis Ritchie.
This is called array-to-pointer conversion. It occurs in cases defined by the C++ standard, causing confusion among beginners. Things are complicated by the fact that the C++ standard doesn't have a single list of cases where such conversion occurs. We have to retrieve them throughout the text, as if we're completing another quest in some Korean MMORPG.
Let's first look at the cases where conversion does not occur.
The discarded-value expression is an expression whose result is not used. No conversion occurs in the example below:
int arr[3] = { 1, 2, 3 };
arr;
As we've already seen at the very beginning of the article, the typeid operator can handle arrays and returns different strings for arrays and pointers.
The same applies to the sizeof operator. It can count sizes of both arrays and pointers. The code below works fine:
int arr[3] = { 1, 2, 3 };
auto ptr = arr;
static_assert(
sizeof(arr) != sizeof(ptr)
);
References. Operations that would otherwise lead to the array-to-pointer conversion would be performed on an array itself when references are used. You can pass an array to a function by reference. You can deduce a reference to an array in a template. Referances are cool. Be like a reference!
Be careful with function overloading! If a compiler has to choose between a function that takes a reference and a function that takes a pointer, it prefers not to choose at all. The code below doesn't compile because both functions fit equally well:
void foo(int (&a)[3]) {};
void foo(int *p) {};
int main()
{
int arr[3] = { 1, 2, 3 };
foo(arr);
}
Back in C days, it was easier. There were (and still are) several cases where the array-to-pointer conversion didn't happen. So, in all other cases, it did. C++ is a slightly more complex language, so you can't get around this rule anymore. Well, let's examine the cases where conversion occurs.
We'll start with operators. All these addition, subtraction, division, multiplication, indexing, and other built-in binary and unary operators can't work with arrays. If you try to add up two arrays, they convert to pointers. This doesn't mean of course that we can add up two pointers. This operation is meaningless at the very least, and at most not part of the standard.
So, if that's how it works with built-in operators, what about user-defined operators?
Overloaded operators, like other functions, take arguments. The standard says that an array-to-pointer conversion is applied to function arguments when an argument is an array. This means that an array itself can't be passed to a function, only a pointer to its element can be.
Even if you write a function parameter as an array:
int foo(int arr[3]);
Inside the function, the arr parameter is a pointer. The array isn't copied.
When using static_cast and reinterpret_cast, we can't convert anything to an array: the output results in a pointer to an element.
Strictly speaking, we can't convert something to an array using const_cast and dynamic_cast either. The other thing is, these two simply don't compile when we try to perform such a conversion.
If the second or third operators of the ternary operator (the one with the question mark!) are arrays, the operator returns a pointer instead of an array.
As far as arrays are concerned, non-type template parameters are similar to function parameters. If we declare a parameter as an array, it turns out to be a pointer, actually:
template <int arr[3]>
void foo();
Arrays used as template arguments in a function call are deducted as pointers. In the example below, the template parameter is deducted as a pointer:
template <typename T>
void foo(T arr) {};
//....
int arr[3] = { 1, 2, 3};
foo(arr);
The same applies to conversion function template. Arrays become pointers:
template <typename T>
struct A {
operator T() {
static int arr[3];
return arr;
}
};
Don't blame it on me. I didn't force you to learn C++.
I doubt you started reading this article for the sake of theoretical research that you, dear reader, have probably already done. However, it was necessary to smoothly guide the narrative to real-life aspects that can unexpectedly pop up in a programmer's text editor, making developers exclaim, "I hate C++ arrays!"
Let's imagine we have two classes. One is the base one, and the other derives from it:
struct B {
char i;
};
struct D : B {
int y;
};
Besides that, there's a function specified somewhere nearby that traverses an array of base class objects:
void foo(B *b, size_t size)
{
for(auto &&el : std::span(b, size)) {
std::cout << el.i << std::endl;
}
}
There's a catch: it takes a pointer and some size. Such a composition gives some creative freedom to a careless programmer who may write something like the following code:
int main()
{
D arr[3] = { {'a', 1}, {'b', 2}, {'c', 3} };
foo(arr, 3);
}
This is definitely UB. But that's not the issue here, or at least not the only one.
In the example above, the next element in the loop isn't calculated correctly. Rather than displaying the next object B::i member, the program displays the padding bytes. They follow the variable and were added by a compiler for alignment purposes. In our compiler, sizeof(B) is one and sizeof(D) is eight.
Now let's consider a case where both of these classes are polymorphic. We add virtual functions to them:
struct B {
char i;
B(char i) : i(i) {};
virtual void print() { std::cout << "BASE " << i << "\n"; }
};
struct D : B {
int y;
D(char i, int y) : B(i), y(y) {};
void print() { std::cout << "DERIVED " << i << " " << y << "\n"; }
};
We can create an example where this change affects how the program looks and behaves. In the linked example, you can see how the virtual keyword alone causes the elements in the loop to be calculated correctly. However, UB remains.
In the example above, the behavior is caused by the implementation of polymorphic classes used in the compiler. A reasonable but not mandatory way to implement polymorphism is to add the vtable pointer to objects of a polymorphic class that points to a table containing various information about an object class. Apparently, in our case, the pointer is added to the end of the structure, causing the compiler to align the entire structure size to that pointer. To do this, the compiler adds 7 padding bytes after the i variable in the B class objects and 3 bytes after the same variable when using the D class objects (since 4 bytes go to the y variable). As a result, the size of both structures becomes the same, and the iteration runs correctly. But if we change the type of the y variable to long, we won't be so lucky anymore.
The big inconvenience here is that the compiler doesn't warn about this, since converting a pointer to a derived class to a pointer to the base class is supported by the language rules. So, we can imagine a case where the code works with one compiler and platform (even though it shouldn't), and crashes under other circumstances. If the function had accepted parameters like std::array or std::span, there would have been no issues.
Let's look at the following code:
#include <iostream>
int main()
{
int arr[3] = {1, 2, 3};
auto sizeof_1 = [arr] {
return sizeof(arr);
};
auto sizeof_2 = [arr = arr] {
return sizeof(arr);
};
auto sizeof_3 = [=] {
return sizeof(arr);
};
std::cout << sizeof_1() << std::endl;
std::cout << sizeof_2() << std::endl;
std::cout << sizeof_3() << std::endl;
}
What do we know about lambda expressions? Well, they have captures. Captures capture (!sic) variables by value (=) or by reference (&). Captures are implemented by the compiler creating a service class where each captured variable is a non-static class field.
If a variable is written without ampersand and yet isn't this variable, it's passed by value. In the code above, all arrays are passed by value. So, they are all cast using the array-to-pointer conversion. This means that the program displays the same number three times.
Alternatively, as we can read on cppreference, lambda data members that are in a capture expression without an initializer go through direct-initialization. If an array is captured, each of its elements is initialized via direct-initialization in the ascending index direction. So, the program displays the same numbers, just not the ones we had in mind.
Or, as we can also read there, if there's an initializer, the variable to be captured is initialized in the way the initializer dictates. The previously declared array can be used to initialize only a variable of the pointer to that array element type. So, when we write [arr = arr] in the capture, the pointer to the first element is still captured, unlike in other ways of capture by value notation.
This little detail is easy enough to overlook, so one may, for example, overwrite the external array elements when the second type (mentioned above) lambda expression is used.
It seems logical, but there are still some mixed feelings. Most importantly, we've found a C++ context where we can still implicitly copy an array without resorting to library functions!
Be careful, programmer: using a regular pointer instead of an array in this context displays the size of the pointer itself in all three cases!
However, enough has been said about array copying. Let's address iterating over it as well.
There are currently two types of iteration in C++: the classic for loop and its range-based version. We can use both to iterate over arrays. Both have known issues with iterating through a pointer.
We'll cover the nuances of using the classic for loop in the next chapter of the article. In this one, however, we'll focus on its range-based little brother. Let's have a quick recap of how it works.
int arr[3] = {1, 2, 3};
for(auto &&element : arr) std::cout << element << std::endl;
Under the hood, it expands into a regular for loop that handles iterators. This structure also works correctly if you create the prvalue array in place of the arr variable in the loop. The temporary array is bound to the forwarding reference and exists until the loop ends.
Some readers may have already started typing in the comments below that the range-for loop example is incorrect because it uses library functions to get iterators. Like, the element parameter would be obtained using the std::begin library function (or std::cbegin, depending on the element const-ness), and the iterator pointing to the array boundary is obtained via std::end (or std::cend). Indeed, these functions have array overloads. But be careful, programmer, because at the same link to the standard, you can read that iterating over arrays doesn't involve iterators—only good old pointers.
At the same time, you will come a cropper trying to replace the array with the pointer. The following code doesn't even compile:
int *ptr = arr;
for(auto &&element : ptr) std::cout << element << std::endl;
If pointers in a range-based loop lead to unbuildable code, using them in its big brother for can be even more nasty.
Let's say we have a multidimensional array—a matrix of integers, for example:
int arr[2][2][2] = { 0, 1, 2, 3, 4, 5, 6, 7 };
Suddenly, we need to iterate over this array to change each value. We know that when indexing an array of the T[N] type, we get back T. In our case, T is int[2][2]. We can also apply the indexing operation to the obtained construction, thus getting an object of the int[2] type and once again finally reaching the desired int. In fact, if we do it via regular for loops, we need three of those.
We also know that the standard guarantees that array elements are arranged one after the other. In fact, the rule applies recursively to all parts of a multidimensional array. All elements of the int[2][2] type are arranged in sequence and within them, all elements of the int[2] type are arranged in sequence, and so on.
Of course, everybody knows that taking this logic too far is dangerous to the program health. The code below is incorrect:
#include <iostream>
int main()
{
int arr[2][2][2] = { 0, 1, 2, 3, 4, 5, 6, 7 };
for(size_t i = 0; i < 8; ++i) {
std::cout << arr[0][0][i] << std::endl;
}
}
Here we try to access elements of the int type via the very first int[2] sub-array of the very first int[2][2] sub-array, thus going beyond no less than three array boundaries. But elements of the int type are still in memory one after the other—the standard ensures it! Indeed, it does, as well as ensuring that the behavior is undefined for the given code.
UB is malum in se. What's worse is that the code may work quite well because the elements are actually arranged in a sequence. Or it may crash pretty hard. If you don't believe us, you can see for yourself.
Great! We know that adhering to the standard is a good thing! Let's write three loops. But what if the array is four-dimensional, though? And what if it's five-dimensional? What if templates come into play and the dimensionality can be whatever you want?
Do the dark forces possess some sorcery after all? Some magic that may allow us to get around the restriction, maybe. Let's rewrite the code as follows:
#include <iostream>
#include <type_traits>
int main()
{
int arr[2][2][2] = { 0, 1, 2, 3, 4, 5, 6, 7 };
auto ptr = reinterpret_cast<std::remove_all_extents_t<decltype(arr)>*>(arr);
for(size_t i = 0; i < 8; ++i) {
std::cout << ptr[i] << std::endl;
}
}
Let's say some evil genius decided to get the type of an element in a multidimensional array using std::remove_all_extents_t. Then they cast that array to a pointer to that element using reinterpret_cast. In fact, such intricacies result in an analog of the flat() function in other programming languages. That function squashes a multi-dimensional array into a one-dimensional one. We even save on pointer arithmetic by adding only one index to ptr instead of three, as in the case of arr.
Unfortunately, this is still UB. In this case, in addition to crossing array boundaries, strict aliasing rules also come into play: we can't access an object using a type other than the one it was created with. Another thing is that when we use the same compiler as in the previous example, the current example not only compiles but also runs without crashing, even with the sanitizer enabled.
Be careful, friends, arbitrary conversions between array types won't keep a doctor away!
Much has already been said about the issues one may face when mixing up an array and a pointer to its first element in the sizeof operator arguments. We won't repeat ourselves and invite you to read more about it here.
We hope you didn't take this article as a criticism of built-in arrays in C. After all, they had their place and their use there, due to the language particularities.
However, we're talking about C++ now. So, it'd be wrong to leave an absolutely logical question unanswered at the end of the article. This question goes something like, "What should we do in C++ with such an irritating feature of the C language?"
Well, we couldn't come up with a better answer than "use std::array or std::span".
The issue with using a pointer to a base class when iterating over an array of derived class objects is solved by using std::array or std::span. The compiler doesn't allow us to push an array of derived elements into an array of base elements.
In "lambdas", we talked about the degenerate case of passing an array to captures by value, where the behavior is different. Again, std::array helps. In all cases where it's passed by value, all elements are fully copied. When it comes to std::span, the elements aren't copied, again, in all three cases.
"Iterating" over std::array and std::span works like a charm. You can choose between the regular for, the range-based one, or library functions.
However, std::array and std::span can't help much with iterating over a multidimensional array. If one tries hard enough, they can make similar mistakes with them too. Moreover, it's a pain in the neck to declare multidimensional std::array and std::span. Well, not every cloud has a silver lining, right? But if we want to declare multidimensional stuff in a compact way, and C++ 23 is already used in the project, we can consider using std::mdspan.
That's it. One should know how to use built-in arrays, but here's a question: are they worth it when C++ has safer alternatives? This question is rather rhetorical, maybe even philosophical. We'll just hope that this short note has made it a little easier for you, dear reader, to answer it!
And if not, feel free to leave your questions in the comments!
Thank you for reading it to the end! El Psy Kongroo.
0