The gaming industry is constantly evolving and is developing faster than a speeding bullet. Along with the growth of the industry, the complexity of development also increases: the code base is getting larger and the number of bugs is growing as well. Therefore, modern game projects need to pay special attention to the code quality. Today we will cover one of the ways to make your code more decent, which is static analysis, as well as how PVS-Studio in practice helps in the game project development of various sizes.
"The most important thing I have done as a programmer in recent years is to aggressively pursue static code analysis. Even more valuable than the hundreds of serious bugs I have prevented with it is the change in mindset about the way I view software reliability and code quality." – John Carmack
We have been working with major game developers for many years and during this time we managed to do a lot of interesting and useful things for the gaming industry. It's not much of a surprise, given the list of our clients from the gaming industry. We actively support our clients: to integrate PVS-Studio into their own development process, fix errors found by the analyzer, and we even make special custom features.
In addition, we do a lot of independent development of the analyzer in the GameDev direction, as well as promote PVS-Studio, telling people about interesting errors that it has found in various video games.
Sure, we do have some interesting stories to tell. This article will cover several such cases.
One of the ways we promote our product is by writing articles about checking open projects. Everyone benefits from these articles: a reader gets the chance to check out some unusual errors in a familiar project and learn something new. As for the PVS-Studio team, we get the opportunity to show the work done on real code, so that project developers can learn about errors and fix them in advance.
Our first major acquaintance with Unity took place in 2016, when the developers of this game engine opened the source code of several components, libraries, and demos in their official repository. No wonder, we could not pass by such an alluring case and wanted to write an article about checking the posted code.
Then we found out that the Unity3D code (at that time the engine was called like that) was of very high quality. But still we were able to find quite a lot of serious errors in it. There were enough of them to write an article.
Two years later, another thing happened – Unity developers opened the code of the engine and the editor itself. And just like the previous time, we could not take no notice of that and checked the source code of the engine. And it was not for nothing - we also found a bunch of captivating flaws.
At the same time our intentions go far beyond just writing articles. We continue to work on PVS-Studio, and GameDev is one of the most significant areas for our development. Therefore, we want Unity game developers to be able to get the best possible analysis of their projects.
One of the steps to improve the quality of Unity projects analysis was writing annotations for methods defined in the Unity Scripting API.
Method annotation is a special mechanism used in PVS-Studio. It allows a user to provide the analyzer with all the necessary information about a particular method. It is written in special code by the analyzer developers themselves (i.e., by us).
This information can be of completely various kinds. For example: how the method can affect the parameters passed to it, whether it can allocate memory, and whether it returns a value that must be handled. Thus, annotation allows the analyzer to better understand the logic of methods, allowing it to detect new and more complex errors.
We have already written a huge number of different annotations (for example, for methods from the System namespace), and we were happy to add method annotations from the Unity Scripting API to them.
We started to extend the list of annotations with an assessment. How many methods are there in total? Which ones should be annotated first? There were a lot of methods in total, so we decided to start by annotating the most frequently used ones.
This is how we were looking for popular methods: first, we gathered a pool of projects from GitHub that use Unity features, and then we used a self-written utility (based on Roslyn) to calculate calls to the methods we were interested in. As a result, we got a list of classes whose methods were used most often:
Next, it remained to annotate the methods of these classes. We created a test project and dug into the documentation to get as much information about those methods as possible. For example, we tried passing null as various arguments to see how the program would behave.
During such checks, we were discovering some interesting undocumented information from time to time. We even found a couple of noteworthy bugs in the engine. For example, when we are running the following code:
MeshRenderer renderer = cube.GetComponent<MeshRenderer>();
Material m = renderer.material;
List<int> outNames = null;
m.GetTexturePropertyNameIDs(outNames);the Unity editor itself crashes (at least in version 2019.3.10f1). Of course, it is unlikely, that anyone will write such code. Still the fact that the Unity editor can be crashed by running such a script is curious.
So, we had the annotations written. After running the analysis, we immediately found new triggerings. For example, the analyzer detected a strange call to the GetComponent method:
void OnEnable()
{
GameObject uiManager = GameObject.Find("UIRoot");
if (uiManager)
{
uiManager.GetComponent<UIManager>();
}
}Analyzer warning: V3010 The return value of function 'GetComponent' is required to be utilized. - ADDITIONAL IN CURRENT UIEditorWindow.cs 22
The GetComponent method implies the return of a specific value even due to its name. It is logical to assume that this value should be used in some way. Now thanks to the new annotation, the analyzer knows that such an "unattended" call to this method may indicate a logical error and warns about it.
This is not the only warning that appeared in the set of our test projects after adding new annotations. I will not cite the rest, so as not to make this article too large. The main thing is that now the development of Unity projects using PVS-Studio allows you to write much safer and cleaner code without bugs.
If you would like to read more about our work with annotations for Unity methods, here is the article: How the PVS-Studio analyzer began to find even more errors in Unity projects.
When, back in 2014, the developers of Unreal Engine 4 opened the source code of the engine, we simply could not get past that project and also wrote an article about it. The engine developers liked the article and fixed the errors we found. But this was not enough for us, and we decided to try to sell the license for our analyzer to Epic Games.
Epic Games was interested in improving its engine with PVS-Studio, so we agreed on the following: we fix the Unreal Engine code on our own so that the analyzer does not issue any warnings, and guys from Epic Games buy our license and additionally reward us for the work done.
Why all warnings had to be fixed? The fact is that one can get the maximum benefit from static analysis by correcting errors right when they appear. When you check your project for the first time, you usually get several hundred (and sometimes thousands) warnings. Among all these analyzer triggerings, it is easy to lose warnings issued for newly written code.
At a first glance, this problem can be solved quite easily: you just need to sit down and go through the entire report, gradually correcting errors. However, although this method is more intuitive, it may take time. It is much more convenient and faster to use suppress files.
Suppress files are a special feature of PVS-Studio that allows you to hide analyzer warnings in a special file. However, hidden warnings will not appear in subsequent logs: you can view them separately.
After having many triggerings after the first check, you can add all detected warning to the suppress file in a couple of clicks, and you will get a clean log without a single entry after the next check.
Now that the old warnings are no longer included in the log, you can easily detect a new warning immediately when it appears. Here's the order of actions: write the code –> check it with the analyzer –> spot a new warning –> fix the error. This is how you will get the most out of using the analyzer.
At the same time, do not forget about the warnings in the suppress file: they can still contain warnings about major errors and vulnerabilities, just as before. Therefore, one should return to these warnings and reduce their number on a regular basis.
No doubts, this scenario is convenient, but developers from Epic Games wanted their code to be fixed straight away, so they passed the task to us.
And we got to work. After checking the project code, we found 1821 warnings of Level_1 and Level_2. Parsing such a large volume of warnings requires serious work, and to facilitate this whole process, we have set up continuous code analysis on our CI server.
It looked like this: every night on our server, the current version of Unreal Engine 4 was built, and immediately after the build, the analysis was automatically started. Thus, when our guys came to work in the morning, they always had a fresh report from the analyzer, which helped them to track the progress of eliminating warnings. In addition, this system allowed us to check the build stability at any time by running it on the server manually.
The whole process took us 17 working days. The schedule for fixing errors was as follows:
In fact, this schedule does not fully reflect our work. After we fixed all warnings, we waited another two days for them to accept our latest pull requests. All this time, the latest version of Unreal Engine was being checked automatically, which, in turn, continued to be updated with new code. So, what do you think happened? During those two days, PVS-Studio found four more errors in the code! One of them was crucial and could potentially lead to undefined behavior.
Of course, we also fixed those errors too. At that point developers of Unreal Engine had only one thing left: set up automatic analysis in their own place, just as we've done. From that moment on, they began to see warnings every day that were issued for the code they had just written. This allowed them to fix errors in the code right when they appeared – at the earliest stages of development.
You can read more about how we worked on the Unreal Engine code in the official Unreal Engine blog or on our website.
Did I mention that we check various open projects and write articles about them? So, we now have a whole lot of similar articles about game projects! We wrote about games like VVVVVV, Space Engineers, Command & Conquer, osu! and even (a very early article) Doom 3. We've also compiled the top 10 of most interesting software bugs from the video game industry.
So, we checked probably most of the well-known open source engines. In addition to Unity and Unreal Engine 4, projects such as Godot, Bullet, Amazon Lumberyard, Cry Engine V and many others have come under our sights.
The best part of all this is that many of the bugs we described were later fixed by the project developers themselves. It's nice to feel that the tool you are developing brings real, visible, and tangible benefits to the world.
You can view a list of all our articles related to video game development in one way or another on a special page of our blog.
At this point my article comes to an end. I wish you clean and correctly working code without bugs and errors!
Interested in the topic of static analysis? Want to check your project for errors? Try PVS-Studio.