Tool to check C++ #include dependencies (dependency graphs created in .dot format)

Related tags

cpp-dependencies
Overview

Read Me for Dependency Checker

Codacy Badge Mac/Linux Build Status Windows Build Status License Release

Copyright (C) 2012-2017, TomTom International BV. All rights reserved.

The tool cpp-dependencies creates #include dependency information for C++ source code files, which it derives from scanning a full source tree.

The dependency information is output as .dot files, which can be visualized in, for example, GraphViz.

Happy coding!

Peter Bindels and Rijn Buve

TomTom International BV

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Build and run

The tool depends on Boost.Filesystem being available and usable. Installing this should be done with your platform's package management system, such as Apt, Pacman or Brew.

The build configuration is created with CMake. To create the build configuration for your build system (GNU make, MSBuild/Visual Studio) create a build directory outside this source directory and run

cmake <PATH_TO_THIS_SOURCE_DIR>

If you want to use Boost::Filesystem instead of std::filesystem, if your platform does not have a std::filesystem implementation yet or if you prefer it, add -DWITH_BOOST to the invocation of CMake.

To build the tool, either execute

make

for GNU make or open the Visual Studio solution file generated in the build directory.

This creates the executable file cpp-dependencies.

To check if the tool was compiled correctly, execute:

./cpp-dependencies

This provides help information about the tool. More information about its usage is presented in the next paragraph.

Using cpp-dependencies to analyze a component

As a first thing on a code base is to find out whether it can read the code correctly. From the root of the project, run the command:

cpp-dependencies --stats .

to determine the complexity of the code base and the amount of nodes that are entangled in cycles with other components. In well set-up projects, the cycle count will be equal to zero and the amount of components will be in the same order of size as the logical components you expect.

To investigate a specific component, you can use

cpp-dependencies --info <component> .

for all information the tool has on the component, or:

cpp-dependencies --inout <component> .

to find out who links to and from your component.

In case you have a dependency that you were not expecting, or find out that when rebuilding component A that a supposedly-unrelated component B is built, you can use:

cpp-dependencies --shortest A B .

to determine why there is a link from component A to component B. It will find one of the shortest paths it can find from A to B if there is one.

Using cpp-dependencies to make visualized graphs

The tool is also able to provide output in .dot format, which is a format used by GraphViz and other tools to contain graphs. It is a human-readable format and can be used by the dot tool to convert it into a graphical image. To create a graph file, use:

cpp-dependencies --graph mygraph.dot .

to create a mygraph.dot file containing the full component graph.

You can restrict the component graph to either all components beneath a given target (--graph-for <output> <target>) or all components part of a cycle (--graph-cycles).

To make this text-format graph into a viewable graph, use for example:

dot -Tpng mygraph.dot >mygraph.png

to convert it into a PNG file.

The dot program will try to find a way to graphically display the graph output. Note that very large graphs, in particular if many cycles are present, can take hours to render.

Example use of cpp-dependencies

In the source tree there's a folder example which contains an empty skeleton project, which does have some dependency information to be extracted from it. To start analyzing it, we first run the tool to extract statistics:

> cpp-dependencies --dir example --stats
6 components with 5 public dependencies, 1 private dependencies
Detected 2 nodes in cycles

This informs us that there is something not quite right with the dependencies. It sees 6 components: the root folder, four libraries and an executable. The simplest way to find out what's wrong is to draw out the graph in a visual way:

> cpp-dependencies --dir example --graph dependencies.dot
> dot -Tpng dependencies.dot >dependencies.png

Then open this PNG file in any tool that can view it, such as a web browser. This shows us the following image:

Dependency graph showing a cycle between Engine and UI

The light blue links are an implementation-only link, the dark blue ones expose some part of this dependency on their interface. The orange ones are the most interesting ones; they are places where a component can reach itself through some other component. Let's find out why this is there:

> cpp-dependencies --dir example --shortest Engine UI
Engine -> UI
  ./Engine/Engine.h includes ./UI/Display.h
> cpp-dependencies --dir example --shortest UI Engine
UI -> Engine
  ./UI/Display.cpp includes ./Engine/Engine.h
  ./UI/Display.h includes ./Engine/Engine.h

At this point, it's up to the developer or architect to find out which of these two dependencies is the wrong way around and to find a way around that. In the example, the Engine component should not be talking directly to the UI component. Removing this dependency results in the following statistics:

> cpp-dependencies --dir example --stats
6 components with 4 public dependencies, 2 private dependencies
Detected 0 nodes in cycles

The cycle has been removed, and there is one less dependency. We can find out what the shortest path is to the DataAccess component from the executable:

> cpp-dependencies --dir example --shortest main DataAccess
main -> UI
  ./main/main.cpp includes ./UI/Display.h
UI -> Engine
  ./UI/Display.cpp includes ./Engine/Engine.h
  ./UI/Display.h includes ./Engine/Engine.h
Engine -> DataAccess
  ./Engine/Engine.cpp includes ./DataAccess/DA.h

This tells us that there's no path shorter than three steps, and it informs us for each step of the way why it detects this link. In more complicated cycles, this can be a way to isolate the thinnest part of the cycle. In situations where there's an invalid dependency from one component to another - for example, from a unit test of one component to a very far away different component, this can help you identify where on the path from A to B a wrong link is present. It can also be used to explicitly verify that a link is not present, such as the one we just removed:

> cpp-dependencies --dir example --shortest Engine UI
No path could be found from Engine to UI

The graph now also shows proper dependency ordering:

> cpp-dependencies --dir example --graph newdependencies.dot
> dot -Tpng newdependencies.dot >newdependencies.png

Dependency graph showing no more cycles

We can regenerate the CMakeLists.txt files as well to remove the dependency from the build inputs, so that our build system will also know that the link is no longer present:

> cpp-dependencies --dir example --dryregen
Difference detected at "./Engine"
> cpp-dependencies --dir example --regen

Customizing the outputs

As cpp-dependencies has a lot of analysis it can do on the source tree, there are also some configurable parts to it. The configuration can be found in the file config-cpp-dependencies.txt that should be in your project root. It allows you to customize the colors used in generation, the thresholds for outlier detection and some minor parameters. Please read the documentation in the example config-cpp-dependencies.txt that is in the source distribution for the tool to see all the options.

Editing the tool

The tool itself is split up into a few separate files to make it easier to find and extend its functionality. The following files are found:

  • main.cpp contains the main functions and help information, as well as the core flow.
  • Input.cpp contains the functions that read C++ and CMakeLists files into the information needed by the tool.
  • Output.cpp contains functions to write all output files generated, except for the CMakeLists generation.
  • CmakeRegen.cpp contains the functionality to write CMakeLists files.
  • Analysis.cpp contains all graph processing and navigation functions.
  • Component.cpp contains the implementation needed for the struct-like data storage classes.
  • generated.cpp contains the function to convert found header files into a lookup map. Also the place to add generated files to the known file list, so that they will be taken into account for components.
  • Constants.h contains the constants used throughout the code base.

In general, the root functionality is kept in main.cpp, the structural classes are kept in Component.cpp and any auxiliary functions that are used to do this are split up by domain.

Rationale behind implementation

The tool was implemented with the goal of being able to quickly analyze dependencies between components of a complex project, including how the dependency graph changes when some changes are made to the source tree. To accomplish this, choices were made in the direction of more performance at the expense of strict correctness. Specifically:

  • It does not use a proper C++ parser to read C++ files, nor a proper CMake parser to read CMake files. Properly parsing these files would increase the full run time of the program by orders of magnitude and make it much less useful.
  • strstr is used across the full code base. While profiling, we found that std::string::find was taking over 80% of the full runtime. Replacing it with strstr, which is typically much more optimized, made the whole program twice as fast.

This results in it running on a 1.5GB source code base in about 2.1 seconds -- fast enough for interactive checks and rerunning after any small modification.

The tool was set up to compile on a Ubuntu 12.04 system with the platform default compiler. This means that the sources will use C++11 but will not use anything not available in GCC 4.6. It has been tested and used on Linux (Ubuntu 12.04 - 16.04) and MacOS X (different versions).

Using Git and .gitignore

It's good practice to set up a personal global .gitignore file on your machine which filters a number of files on your file systems that you do not wish to submit to the Git repository. You can set up your own global ~/.gitignore file by executing: git config --global core.excludesfile ~/.gitignore

In general, add the following file types to ~/.gitignore (each entry should be on a separate line): *.com *.class *.dll *.exe *.o *.so *.log *.sql *.sqlite *.tlog *.epoch *.swp *.hprof *.hprof.index *.releaseBackup *~

The local .gitignore file in the Git repository itself to reflect those file only that are produced by executing regular compile, build or release commands.

Bug reports and new feature requests

If you encounter any problems with this library, don't hesitate to use the Issues session to file your issues. Normally, one of our developers should be able to comment on them and fix.

Issues
  • Removing dependency from boost

    Removing dependency from boost

    It's great idea of cpp-dependencies. I think it will be very usefull. I download cpp-dependencies and try to build it at windows, but I haven't got configured boost for it. So I decide remove dependency from boost. It's increase requirements for compiler to vc140 and gcc 5.3, but in cpp-dependencies already used C++11, so I think it possible. Please if you think it will be useful make pull-requests.

    p.s. It my first pull-request at github, so if I made something wrong please correct me.

    opened by ChernovAO 10
  • Be nice to clearly state that you need to be using CMake

    Be nice to clearly state that you need to be using CMake

    The README didn't make it clear to me that it needs CMakeLists.txt to do anything useful. I managed to get it building on WIndows but our code uses MSVC projects so all I got was:

    1 components with 0 public dependencies, 0 private dependencies
    Detected 0 nodes in cycles
    
    opened by toby-allsopp 7
  • Error when compiling with VS on Windows - memrchr

    Error when compiling with VS on Windows - memrchr

    memrchr() function is a GNU extension. Input.cpp in ReadCodeFrom()

    opened by ipostr08 5
  • Add CMake build configuration for cpp-dependencies itself

    Add CMake build configuration for cpp-dependencies itself

    I was annoyed by the fact that I had to modify and patch the Visual Studio solution/project files manually each time I tested cpp-dependencies on a different machine or did a clean git checkout. I had to adjust the paths to Boost in two places (include path and library path) as well as I had to switch the toolset (v120 is pre-selected in the solution file, I've only v140 installed) and consequently had to modify the Boost library names.

    I've written simple CMakeLists files to let CMake generate the build instructions (GNU make, MSBuild/Visual Studio). The location of Boost::filesystem (and the implicitly dependent Boost::system) is detected by CMake. People having Boost in a non-default location can use -DBOOST_ROOT=<BOOST_PREFIX_PATH> to tell CMake to preferably search in the given path for Boost headers and libraries. With an optional argument -DWITH_TESTS=OFF one can skip the creation of the unittest target.

    To ease building the unit tests, I've added GTest as a Git submodule and link the unittest target to the gtest_main target.

    The following targets are created for Debug, Release and RelWithDebInfo configurations:

    • cpp_dependencies_obj all source files, except main.cpp
    • cpp-dependencies the actual executable
    • if WITH_TESTS=ON (the default)
      • gtest_main GoogleTest static library
      • unittests cpp-dependencies' unittests

    I have deleted the existing build scripts and solution files.

    I have tested this

    • on a Linux machine (openSUSE 42.1) with CMake 3.6.1, GCC 6.2.1 and Boost 1.54.0
    • on a Windows 7 64bit machine with CMake 3.5, Visual Studio 2015 Update 3 (v140 toolchain) and Boost 1.61.0

    ToDo:

    • [ ] adjust Travis configuration
    opened by torbjoernk 5
  • Added cmake requirement for C++11

    Added cmake requirement for C++11

    Following the cmake documentation a line declaring C++11 language standard requirement was added. This tells cmake to test the compiler for the language feature, and set the compiler parameters required for the language standard.

    opened by cceelen 5
  • --ambiguous includes a symlink to a file as ambiguous to that file

    --ambiguous includes a symlink to a file as ambiguous to that file

    In my test case './include/a.h' is a symbolic link to './src/a.h'. I am getting the following message with '--ambiguous'. I assume there should not be any message.

    Found 1 ambiguous includes
    
    Include for a.h
    Found in : 
      included from ./src/a.cpp
    Options for file: 
      ./include/a.h
      ./src/a.h
    
    
    opened by snandan 4
  • CMakeLists.txt files in subdirectories are unconditionally included in analysis

    CMakeLists.txt files in subdirectories are unconditionally included in analysis

    CMakeLists.txt files in subdirectories, like those from submodules, are included in analysis regardless of whether they are in the transitive-include graph of the top-level CMakeLists.txt file.

    src/Input.cpp L 267 will pull in any file named CMakeLists.txt.

    This results in misdiagnosis of ambiguous includes in projects like https://github.com/jbcoe/polymorphic_value which include the popular Catch testing framework.

    opened by jbcoe 4
  • Revisit CI configuration

    Revisit CI configuration

    AppVeyor

    Fix the AppVeyor config to use the right logger, which updates the Web UI with warnings and errors generated during the build. Here is a sample. Upload the built cpp-dependencies installer as an artifact. Here is a sample. TODO: check if the cpp-dependencies standalone executable needs to be added as another artifact.

    Travis

    Fix the unused variable CMAKE_C_COMPILER warning from CMake. EDIT: reverted the language changes in Travis config.

    CMake

    Remove the WITH_TESTS option in favor of the conventional BUILD_TESTING option added by CTest. Add a BUILD_COVERAGE option which changes the compile flags of cpp_dependencies_lib, rather than compiling it twice every time.

    • This addresses the warnings from AppVeyor (MSVC) about the unused compiler flags.
    • It is OFF by default, and is not used by any builds. This can be used later to integrate with a coverage visualization service - codecov or coveralls.
    • Reduces the redundant mention of source files (and one target) in CMakeLists.txt.

    I've kept these changes as independent commits. The ones required can be cherry-picked. If there are concerns, please let me know. I'll try to address them.

    opened by tusharpm 3
  • Support Windows

    Support Windows

    The tool so far has not been tried or tested on Windows.

    opened by dascandy 3
  • Support Mac OS X

    Support Mac OS X

    In Input.cpp, the Linux function memrchr() is used. This is a non-standard function, and should be east to replace by something more portable.

    opened by mbitsnbites 3
  • remove experimental deprecation with c++17 in msvc build

    remove experimental deprecation with c++17 in msvc build

    It seemd without it i have an #error from MSVC

    [build] C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30037\include\experimental/filesystem(30,1): fatal error C1189: #error: The <experimental/filesystem> header providing std::experimental::filesystem is deprecated by Microsoft and will be REMOVED. It is superseded by the C++17 <filesystem> header providing std::filesystem. You can define _SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING to acknowledge that you have received this warning. [D:\p4\cpp-dependencies\build\src\cpp_dependencies_lib.vcxproj]

    opened by simdax 0
  • Assertion failed in src/Input.cpp:304

    Assertion failed in src/Input.cpp:304

    final level of parentheses=-1
    cpp-dependencies: ../src/Input.cpp:304: void ReadCmakelist(const Configuration&, std::unordered_map<std::__cxx11::basic_string<char>, Component*>&, const boost::filesystem::path&): Assertion `parenLevel == 0 || (printf("final level of parentheses=%d\n", parenLevel), 0)' failed.
    Aborted (core dumped)
    
    opened by Warchant 2
  • C++17 std::filesystem instead of boost

    C++17 std::filesystem instead of boost

    Can we use std::filesystem (in c++17) instead of Boost::filesystem?

    opened by paamand 2
  • Does it support bazel build system ?

    Does it support bazel build system ?

    I am using Bazel build system for my c++ project and i was trying to generate the dependency graphs for the project. After running the tool, it showed me this error:

    Warning: Analyzing your project resulted in a very low amount of components. This either points to a small project, or
    to cpp-dependencies not recognizing the components.
    
    It tries to recognize components by the existence of project build files - CMakeLists.txt, Makefiles, MyProject.vcxproj
    or similar files. If it does not recognize any such files, it will assume everything belongs to the project it is
    contained in. You can invert this behaviour to assume that any code file will belong to a component local to it - in
    effect, making every folder of code a single component - by using the --infer option.
    
    Another reason for this warning may be running the tool in a folder that doesn't have any code. You can either change
    to the desired directory, or use the --dir <myProject> option to make it analyze another directory.
    
    opened by rahul1990gupta 1
  • std::bad_alloc (core dumped) on Ubuntu 16.04

    std::bad_alloc (core dumped) on Ubuntu 16.04

    Build went fine on Ubuntu 16.04 But execution failed /usr/local/bin/cpp-dependencies --stats . terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc Aborted (core dumped)

    opened by AlbanAndrieu 0
  • Add support for d3.js visualisations

    Add support for d3.js visualisations

    @dascandy I'll make a start on this soon, been a tad busy and been unable to work on personal stuff lately.

    opened by shreyasbharath 0
  • Release pre-built binaries for Linux/MacOS/Windows

    Release pre-built binaries for Linux/MacOS/Windows

    Hi,

    I think this is a great tool and it would be even better if pre-built binaries were packaged up as releases (can be done automatically by the CI when a new release is made).

    I personally feel it is unreasonable to expect every user to compile the tool from source in order to use it.

    Thanks.

    opened by shreyasbharath 1
  • No such file or directory error

    No such file or directory error

    I am getting this error

    cpp-dependencies-1.1/example/Engine/./Engine.h:1:33: fatal error: Framework/framework.h: No such file or directory #include "Framework/framework.h"

    when calling "make" after "cmake example"

    What am I doing wrong can you help me?

    opened by mertbahadir 2
  • Make very explicit what the tool does with CMakeLists

    Make very explicit what the tool does with CMakeLists

    I started using the tool on my codebase recently and for some components I could make some improvements already.

    But with bigger contexts the tool still struggles. I tried to find out why. As far as I can see, the tool is not able to handle the INCLUDE() Cmake directive correctly, so some crucial targets are missing and my depedency graph is wonky.

    Is this support indeed missing? Where would on start adding it?

    opened by pizzard 3
  • Additional target parameters

    Additional target parameters

    This will allow users freedom to specify additional target parameters in add_executable() and add_library() in more relaxed way, e.g:

    add_library(MY_PROJECT_1 SHARED EXCLUDE_FROM_ALL my_source_1.cpp)
    add_library(MY_PROJECT_2
      SHARED
      EXCLUDE_FROM_ALL
      my_source_2.cpp
    )
    

    Next it will no longer enforce the usage of STATIC libraries by default after regeneration, but leave the possibility for the user to leave it blank and obtain then then the default setting of CMake which is STATIC anyway. That behaviour can in turn then easily be adjusted by using the CMake flag BUILD_SHARED_LIBS.

    opened by maikelvdh 0
Releases(1.1)
  • 1.1(Nov 26, 2016)

    This is a fairly major non-breaking change:

    Functionality:

    • Added IncludeSize function, that gives an indication which includes cause your compiler the most grief.
    • Use CMake for the build instead of Makefiles / VS project files.
    • Enable use of std::filesystem, if you have a compiler that has it.

    Speed:

    • More accurate and faster parsing logic.
    • Now using Trajan's algorithm for finding cycles, which makes the cycle analysis much faster. Saves 20% on total runtime on my test target.
    • Enabled use of mmap for reading files.

    Code quality:

    • Fixed various bugs, all resulting in crashes in case of misuse.
    • Fix issues found by PVS-Studio. 19 minors and one critical.
    • Add Appveyor build & Travis build with unit tests, running on Linux, Mac and Windows.
    • Switch testing to simple test framework, to avoid gtest's use troubles on Travis and Appveyor.
    Source code(tar.gz)
    Source code(zip)
  • 1.0(Aug 24, 2016)

Owner
TomTom
TomTom is the location technology provider enabling developers to build applications which are contributing to a safer, cleaner and less congested world.
TomTom
A tool for use with clang to analyze #includes in C and C++ source files

Include What You Use For more in-depth documentation, see docs. Instructions for Users "Include what you use" means this: for every symbol (type, func

null 2.5k Oct 15, 2021
CITL's static analysis engine for native code artifacts

citl-static-analyzer Fast binary hardening analysis tooling. Building on Linux The build process varies by Linux distribution, owing to differences be

Cyber Independent Testing Lab 14 Sep 27, 2021
Pharos Static Binary Analysis Framework

Automated static analysis tools for binary programs

Software Engineering Institute 1.1k Oct 19, 2021
Static analyzer for C/C++ based on the theory of Abstract Interpretation.

IKOS IKOS (Inference Kernel for Open Static Analyzers) is a static analyzer for C/C++ based on the theory of Abstract Interpretation. Introduction IKO

NASA - Software V&V 1.4k Oct 19, 2021
Static code checker for C++

cpplint - static code checker for C++ Cpplint is a command-line tool to check C/C++ files for style issues following Google's C++ style guide. Cpplint

null 889 Oct 9, 2021
A static analyzer for Java, C, C++, and Objective-C

Infer Infer is a static analysis tool for Java, C++, Objective-C, and C. Infer is written in OCaml. Installation Read our Getting Started page for det

Facebook 12.7k Oct 18, 2021