To get a trial key
fill out the form below
Team License (standard version)
Enterprise License (extended version)
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Request our prices
New License
License Renewal
--Select currency--
USD
EUR
GBP
RUB
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
To get the licence for your open-source project, please fill out this form
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
To get the licence for your open-source project, please fill out this form
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
I am interested to try it on the platforms:
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Message submitted.

Your message has been sent. We will email you at


If you haven't received our response, please do the following:
check your Spam/Junk folder and click the "Not Spam" button for our message.
This way, you won't miss messages from our team in the future.

>
>
Commit and Pull Request analysis in Tra…
Analyzer Diagnostics
General Analysis (C++)
General Analysis (C#)
General Analysis (Java)
Diagnosis of micro-optimizations (C++)
Diagnosis of 64-bit errors (Viva64, C++)
MISRA errors
AUTOSAR errors
Additional information
Contents

Commit and Pull Request analysis in Travis CI, Buddy and AppVeyor

Apr 06 2021

This article describes how to check Pull Requests on Linux and macOS. To learn about checking Pull Requests on Windows, use documentation for the PVS-Studio_Cmd and CLMonitor utilities (the "Specification of individual files for analysis" section).

Pull Request analysis is available only under PVS-Studio's Enterprise license. You can request a trial Enterprise license at the downloads page.

File list checking mode

Starting with version 7.04, PVS-Studio has a source file list checking mode for Linux and macOS. It works for projects whose build system can generate the 'compile_commands.json' file. The analyzer uses it to extract information about specific files' compilation. If your build system does not support the 'compile_commands.json' file generation, you can try using the Bear utility to generate this file.

You can also use the file list checking mode together with 'strace' (pvs-studio-analyzer trace) - a log that contains intercepted compiler invocations. To do this, you can build the entire project in trace mode. This way, the analyzer gets complete information about compilation flags for all files.

However, this approach has a significant drawback. Each time you run an analysis, you will need to build an entire project and do its complete trace. This contradicts the idea of a quick commit check. Alternatively, you can cache the trace result. But if you change source file dependency structure (for example, add a new #include to a source file), this can make subsequent analyzer runs incomplete.

This is why, if you need to check commits or Pull Requests, we do not recommend combining the file list checking mode with the trace log feature. If, when checking a commit, you can do an incremental build, do look into the incremental analysis mode.

In the file list checking mode, a list of source files is stored to a text file. Then you can use the '-S' parameter to pass this data to the analyzer.

pvs-studio-analyzer analyze ... -f build/compile_commands.json -S check-list.txt

This file contains relative or absolute file paths. Each path must start on a new line. The analyzer ignores any text that is not a source code file path. You can use this specificity to add comments when indicating files manually. However, often the file list is generated during analysis in CI. Thus, the list may include files from a commit or a Pull Request.

At the start of a project's first analysis, the analyzer generates a cache file that is required for the file checking mode to operate correctly. The file contains dependencies of the project's all source files from header files. The need for this file is a peculiarity of C/C++ file analysis. In the future, the file with dependencies will be cached and the analyzer will update it automatically.

To generate or update the dependency file without starting the analysis, use the '--regenerate-depend-info' flag with the 'skip-analysis' key:

pvs-studio-analyzer analyze ... -f build/compile_commands.json \ 
                                --regenerate-depend-info skip-analysis

To analyze a list of files or an entire project, and force dependency cache update, use the '--regenerate-depend-info' flag with the 'run-analysis' key:

pvs-studio-analyzer analyze ... -S source_files \
                                -f build/compile_commands.json \
                                --regenerate-depend-info run-analysis

Now you can use the file list checking mode to quickly check new code before you commit it to the master development branch. In order for the system to recognize the analyzer's messages, the 'plog-converter' utility provides the '--indicate-warnings' flag:

plog-converter ... --indicate-warnings ... -o /path/to/report.tasks ...

When this flag is set and the analyzer's report contains warnings, the converter returns a non-zero code. You can use the return code to block the pre-commit hook, commit or Pull Request. You send the analyzer's generated report by email, or show it on the screen.

General principles of Pull Request analysis

It may take a long time to check an entire project. Usually, it is a good idea to check the project's select part. The main problem is separating new files from the project's other files.

Take a look at this at this sample commit tree with two branches:

PullRequestAnalysis/image1.png

Let's say, most of the code from commit 'A1' has already been checked. A little earlier, a developer created the 'hotfix' branch from the 'A1' commit and changed some files.

As we see from the example above, after 'A1', two more changes were committed to the 'master' branch. As soon as the 'hotfix' branch is ready to be merged, the developer opens a Pull Request to merge 'B3' and 'A3' changes.

You can check the entire result of merging two branches. This approach is slow and not justified, because only a few files were modified. It's more efficient to check only them.

To do this, when in the HEAD of the branch that you intend to merge into 'master', you need to get the diffs between the branches:

git diff --name-only HEAD origin/$MERGE_BASE > .pvs-pr.list

We will discuss the '$MERGE_BASE' variable further in this article. Not every CI service provides the necessary information about the diffs, so in each scenario finding new ways to get this data may be necessary. Below we will provide a detailed description for each described web-service.

After you get the diffs, or to be precise, a list of files that were changed, pass the '.pvs-pr.list' file to the analyzer:

pvs-studio-analyzer analyze -j8 \
                            -o PVS-Studio.log \
                            -S .pvs-pr.list

After the analysis convert the log file (PVS-Studio.log) to a convenient format:

plog-converter -t errorfile PVS-Studio.log --cerr -w

This command will output warnings into stderr (the standard error message output stream).

In addition to listing errors, you can notify the build and test service about these problems. To do this, add the '-W' ('--indicate-warnings') flag to 'plog-converter'. If there is at least one warning, the 'plog-converter' utility return code will change to 2, which, in its turn, will tell the CI service about potential errors in the Pull Request files.

Travis CI

Travis CI stores its configuration in the '.travis.yml' file. For convenience, create a separate bash-script with functions that will be called from the '.travis.yml' file ('bash script_name.sh function_name').

Add the necessary code to the 'bash' script. Indicate the following in the 'install' section:

install:
  - bash .travis.sh travis_install

Open the '.travis.sh' file and add installing the analyzer to the 'travis_install()' function:

travis_install() {
  wget -q -O - https://files.viva64.com/etc/pubkey.txt \
    | sudo apt-key add -
  sudo wget -O /etc/apt/sources.list.d/viva64.list \
    https://files.viva64.com/etc/viva64.list
  
  sudo apt-get update -qq
  sudo apt-get install -qq pvs-studio 
}

Add the analysis launch to the 'script' section:

script:
  - bash .travis.sh travis_script

And to the bash-script:

travis_script() {
  pvs-studio-analyzer credentials $PVS_USERNAME $PVS_KEY
  
  if [ "$TRAVIS_PULL_REQUEST" != "false" ]; then
    git diff --name-only origin/HEAD > .pvs-pr.list
    pvs-studio-analyzer analyze -j8 \
                                -o PVS-Studio.log \
                                -S .pvs-pr.list \
                                --disableLicenseExpirationCheck
  else
    pvs-studio-analyzer analyze -j8 \
                                -o PVS-Studio.log \
                                --disableLicenseExpirationCheck
  fi
  
  plog-converter -t errorfile PVS-Studio.log --cerr -w
}

Run this code after the project is built, for example, after you've built your project on CMake:

travis_script() {
  CMAKE_ARGS="-DCMAKE_EXPORT_COMPILE_COMMANDS=On ${CMAKE_ARGS}"
  cmake $CMAKE_ARGS CMakeLists.txt
  make -j8
}

This is the result:

travis_script() {
  CMAKE_ARGS="-DCMAKE_EXPORT_COMPILE_COMMANDS=On ${CMAKE_ARGS}"
  cmake $CMAKE_ARGS CMakeLists.txt
  make -j8
  
  pvs-studio-analyzer credentials $PVS_USERNAME $PVS_KEY
  
  if [ "$TRAVIS_PULL_REQUEST" != "false" ]; then
    git diff --name-only origin/HEAD > .pvs-pr.list
    pvs-studio-analyzer analyze -j8 \
                                -o PVS-Studio.log \
                                -S .pvs-pr.list \
                                --disableLicenseExpirationCheck
  else
    pvs-studio-analyzer analyze -j8 \
                                -o PVS-Studio.log \
                                --disableLicenseExpirationCheck
  fi
  
  plog-converter -t errorfile PVS-Studio.log --cerr -w
}

Travis CI independently declares the '$TRAVIS_PULL_REQUEST' and '$TRAVIS_BRANCH' environment variables:

  • '$TRAVIS_PULL_REQUEST' stores the Pull Request number, or, if it is a regular branch, the value is 'false';
  • '$TRAVIS_REPO_SLUG ' stores the project repository name.

The diagram below demonstrates how the function operates:

PullRequestAnalysis/image2.png

Travis CI can recognize return codes. Thus, the presence of warnings will instruct the service to tag the commit as one with errors.

Let's take a closer look at this line of code:

git diff --name-only origin/HEAD > .pvs-pr.list

Travis CI merges branches automatically during Pull Request analysis:

PullRequestAnalysis/image3.png

In this case, the analyzer processes 'A4' instead of 'B3->A3'. This specificity makes it necessary to calculate the difference with 'A3' that is at the top of the branch from 'origin'.

The last important detail is caching dependencies of header files from compiled translation units (*.c, *.cc, *.cpp etc). During the first run in the file list checking mode, the analyzer calculates these dependencies and stores them to the '.PVS-Studio' directory. Travis CI allows to cache folders, which is why we'll save the '.PVS-Studio' directory data:

cache:
  directories:
    - .PVS-Studio/

Add this code to the '.travis.yml' file. The '.PVS-Studio' directory stores various data collected after the analysis. This data significantly speeds up subsequent runs of the file list or incremental analysis. If you do not cache dependencies, the analyzer will process all files every time.

Buddy

Like Travis CI, Buddy enables you to automatically build and test projects stored on GitHub. In contrast to Travis CI, Buddy is configured in the web interface (bash support is available), so you do not need to store configuration files in the project.

First, add a new action to the pipeline:

PullRequestAnalysis/image4.gif

Specify the compiler you used to build the project. Note the docker container, installed in this step. For example, there's a special container for GCC:

PullRequestAnalysis/image5.png

Now install PVS-Studio and the necessary utilities:

PullRequestAnalysis/image6.gif

Add the following lines to the editor:

apt-get update && apt-get -y install wget gnupg jq

wget -q -O - https://files.viva64.com/etc/pubkey.txt | apt-key add -
wget -O /etc/apt/sources.list.d/viva64.list \
  https://files.viva64.com/etc/viva64.list

apt-get update && apt-get -y install pvs-studio

Now switch to the 'Run' tab (the first icon). Add the following code to the editor:

pvs-studio-analyzer credentials $PVS_USERNAME $PVS_KEY

if [ "$BUDDY_EXECUTION_PULL_REQUEST_NO" != '' ]; then
  PULL_REQUEST_ID="pulls/$BUDDY_EXECUTION_PULL_REQUEST_NO"
  MERGE_BASE=`wget -qO - \
    https://api.github.com/repos/${BUDDY_REPO_SLUG}/${PULL_REQUEST_ID} \
    | jq -r ".base.ref"`

  git diff --name-only HEAD origin/$MERGE_BASE > .pvs-pr.list
  pvs-studio-analyzer analyze -j8 \
                              -o PVS-Studio.log \
                              --disableLicenseExpirationCheck \
                              -S .pvs-pr.list
else
  pvs-studio-analyzer analyze -j8 \
                              -o PVS-Studio.log \
                              --disableLicenseExpirationCheck
fi

plog-converter -t errorfile PVS-Studio.log --cerr -w

If you read the Travis-CI section, you already know this code. However, now there's a new step:

PullRequestAnalysis/image7.png

Buddy does not analyze the result of branch merging. It analyzes the HEAD of the Pull Request's branch:

PullRequestAnalysis/image1.png

This is why if your commit is 'B3', you need to compare it to 'A3' and get the diffs:

PULL_REQUEST_ID="pulls/$BUDDY_EXECUTION_PULL_REQUEST_NO"
  MERGE_BASE=`wget -qO - \
    https://api.github.com/repos/${BUDDY_REPO_SLUG}/${PULL_REQUEST_ID} \
    | jq -r ".base.ref"`
git diff --name-only HEAD origin/$MERGE_BASE > .pvs-pr.list

To define 'A3', use GitHub's API:

https://api.github.com/repos/${USERNAME}/${REPO}/pulls/${PULL_REQUEST_ID}

Buddy provides the following environment variables:

  • '$BUDDY_EXECUTION_PULL_REQEUST_NO' - the Pull Request number;
  • '$BUDDY_REPO_SLUG' - a combination of the user name and the repository name (for example, max/test).

Use the button below to save the changes, and start Pull Request analysis:

PullRequestAnalysis/image8.gif

As opposed to Travis CI, there is no need to specify the '.PVS-Studio' folder for caching, because buddy automatically caches all files for subsequent runs. So the last step is to save your username and password for PVS-Studio in Buddy. After saving changes, go back to the 'Pipeline', open variable settings and add the login and key for PVS-Studio:

PullRequestAnalysis/image9.gif

After this, adding a new Pull Request or commit will run the analysis. If the commit contains errors, Buddy will indicate this on the Pull Request page.

AppVeyor

AppVeyor, same as Buddy, is configured in the web interface. You do not need to add the '*.yml' file into the project repository.

In the project overview, switch to the 'Settigs' tab:

PullRequestAnalysis/image10.png

Scroll the page down and check "Save build cache in Pull Requests":

PullRequestAnalysis/image11.png

Now choose the 'Environment' tab. Indicate the image used for the build and specify required environment variables:

PullRequestAnalysis/image12.png

If you read previous sections, you already know these two variables - 'PVS_KEY' and 'PVS_USERNAME'. AppVeyor uses them to check the PVS-Studio license.

Scroll the page down and set the cached directory:

PullRequestAnalysis/image13.png

If you do not specify this setting, the analyzer will process the entire project instead of a few files, while the output will contain information only on the specified files. This is why it's important to enter a correct directory name.

Then proceed to writing the script that manages the project's analysis. Open the 'Tests' tab and select 'Script':

PullRequestAnalysis/image14.png

Paste the following code into the 'Test script' textbox:

sudo apt-get update && sudo apt-get -y install jq

wget -q -O - https://files.viva64.com/etc/pubkey.txt \
  | sudo apt-key add -
sudo wget -O /etc/apt/sources.list.d/viva64.list \
  https://files.viva64.com/etc/viva64.list

sudo apt-get update && sudo apt-get -y install pvs-studio

pvs-studio-analyzer credentials $PVS_USERNAME $PVS_KEY

PWD=$(pwd -L)
if [ "$APPVEYOR_PULL_REQUEST_NUMBER" != '' ]; then
  PULL_REQUEST_ID="pulls/$APPVEYOR_PULL_REQUEST_NUMBER"
  MERGE_BASE=`wget -qO - \
    https://api.github.com/repos/${APPVEYOR_REPO_NAME}/${PULL_REQUEST_ID} \
    | jq -r ".base.ref"`

  git diff --name-only HEAD origin/$MERGE_BASE > .pvs-pr.list
  pvs-studio-analyzer analyze -j8 \
                              -o PVS-Studio.log \
                              --disableLicenseExpirationCheck \
                              --dump-files --dump-log pvs-dump.log \
                              -S .pvs-pr.list
else
  pvs-studio-analyzer analyze -j8 \
                              -o PVS-Studio.log \
                              --disableLicenseExpirationCheck
fi

plog-converter -t errorfile PVS-Studio.log --cerr -w

Pay special attention to the following code fragment:

PWD=$(pwd -L)
if [ "$APPVEYOR_PULL_REQUEST_NUMBER" != '' ]; then
  PULL_REQUEST_ID="pulls/$APPVEYOR_PULL_REQUEST_NUMBER"
  MERGE_BASE=`wget -qO - \
   https://api.github.com/repos/${APPVEYOR_REPO_NAME}/${PULL_REQUEST_ID} \
   | jq -r ".base.ref"`

  git diff --name-only HEAD origin/$MERGE_BASE > .pvs-pr.list
  pvs-studio-analyzer analyze -j8 \
                              -o PVS-Studio.log \
                              --disableLicenseExpirationCheck \
                              --dump-files --dump-log pvs-dump.log \
                              -S .pvs-pr.list
else
  pvs-studio-analyzer analyze -j8 \
                              -o PVS-Studio.log \
                              --disableLicenseExpirationCheck
fi

Note how the '$PWD' variable is assigned the 'pwd' command's value. The procedure may seem unusual, but it facilitates the analyzer's correct operation. If you do not set this variable, it will contain a value AppVeyor had specified for its internal purposes.

Then follows the algorithm you already know:

PullRequestAnalysis/image15.png

Now take a look at this code fragment:

PULL_REQUEST_ID="pulls/$APPVEYOR_PULL_REQUEST_NUMBER"
MERGE_BASE=`wget -qO - \
  https://api.github.com/repos/${APPVEYOR_REPO_NAME}/${PULL_REQUEST_ID} \
  | jq -r ".base.ref"`

This code gets diffs for branches involved in the Pull Request. The code uses the following environment variables:

  • '$APPVEYOR_PULL_REQUEST_NUMBER' - the Pull Request number;
  • '$APPVEYOR_REPO_NAME' - the user name and the repository name.

This website uses cookies and other technology to provide you a more personalized experience. By continuing the view of our web-pages you accept the terms of using these files. If you don't want your personal data to be processed, please, leave this site.
Learn More →
Accept