Bug report for the impatient.

Have you ever had one of those nights? You know the ones where you sit down to do the coding you really want to do. Not your day job stuff. No no, the good stuff. That side project you’ve been thinking about all week. You sit down and spend a few uninterrupted hours working in sweet bliss, then head to bed happy with yourself.

This is not one of those nights.

Shiny Object

The project I wanted to work on, “rtek”, uses CMake. At the time of writing, it is both a very popular and confusing build system for C++. So it was a natural choice for my C++ project; well tested, lots of resources to check if things go south1.

One of my goals of rtek was to continue building my own C++ standard library along side it called “sdslib”. To make it usable in other projects, I have it in a separate repository that is a dependency of rtek.

Since I already had a copy of sdslib on my machine, I wanted to use that instead of cloning another copy for rtek. Enter FETCHCONTENT_SOURCE_DIR_<ucName>. If FETCHCONTENT_SOURCE_DIR_<ucName> is set, CMake uses the provided path as the source of the dependency ‘ucName’ instead of cloning it. So, build away:

> cmake -S. -B./build -G"NMake Makefiles" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DFETCHCONTENT_SOURCE_DIR_SDSLIB="../sdslib-cpp"
> cmake --build build
...
NMAKE : fatal error U1073: don't know how to make '..\_deps\sdslib-build\CMakeFiles\sdslib.dir\all'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX64\x64\nmake.exe"' : return code '0x2'
Stop.

Or not. Wrong command? It works without FETCHCONTENT_SOURCE_DIR_SDSLIB. Something is not right. To CMakeLists.txt!

To The Build Configuration!

FETCHCONTENT_SOURCE_DIR_SDSLIB has to do with the FetchContent module. Here are the parts of rtek’s CMakeLists.txt that are related to sdslib:

# rtek/CMakeLists.txt

FetchContent_MakeAvailable(sdslib)
    sdslib
    GIT_REPOSITORY "https://github.com/sdsmith/sdslib-cpp.git"
    GIT_TAG origin/master
    GIT_PROGRESS 1
    UPDATE_COMMAND ""
    PATCH_COMMAND ""
    TEST_COMMAND ""
)

set(SDSLIB_BUILD_TESTS OFF CACHE BOOL "sdslib build tests")
FetchContent_MakeAvailable(sdslib)
set_target_properties(sdslib PROPERTIES
    MAP_IMPORTED_CONFIG_DEVELOP Debug
)

add_executable(${PROJECT_NAME} ${all_code_files})
add_dependencies(${PROJECT_NAME} spdlog fmt glfw glad sdslib)

target_include_directories(${PROJECT_NAME} PUBLIC
    "${CMAKE_CURRENT_SOURCE_DIR}/src"
)

target_link_libraries(${PROJECT_NAME} PRIVATE
    spdlog::spdlog
    fmt::fmt
    glfw
    glad
    sdslib
)

Nothing looks out of the ordinary. I can build sdslib independently. It must be something to do with rtek’s CMakeLists.txt.

But first, let’s take a step back and experiment to narrow down the cause.

It could be a bad CMake cache. I’ll do a fresh build without any dependencies (saved in _deps) or build artifacts (saved in build).

> rm -rf _deps build   # <<<< clean build
> cmake -S. -B./build -G"NMake Makefiles" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DFETCHCONTENT_SOURCE_DIR_SDSLIB="../sdslib-cpp"
> cmake --build build
...
NMAKE : fatal error U1073: don't know how to make '..\_deps\sdslib-build\CMakeFiles\sdslib.dir\all'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\HostX64\x64\nmake.exe"' : return code '0x2'
Stop.

It still reproduces the problem, it isn’t a fluke. I don’t know what else to do but dive into the generated files and try to reverse engineer whats wrong to fix my CMakeLists.txt.

I have experience debugging Makefiles, but I’m not sure how close NMake is to Make. I don’t have make installed on this machine. I am familiar with Visual Studio (VS) and that’s installed. So let’s build that and poke around.

> cmake -S. -B./build -G"Visual Studio 16 2019" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DFETCHCONTENT_SOURCE_DIR_SDSLIB="../sdslib-cpp"

Opening up the generated VS solution and building it: no error! This is good! It narrows down the problem space.

To make sure it actually does build, let’s blow away the cache and build with the VS and NMake a couple times. Yup, always passes with VS and fails with NMake. It looks like the NMake generator is bugged.

CMake Generators

CMake is a tool that generates build configurations that actually build your project: cmake -> build system -> build output. What build tool is generated is decided by the generator.

There are three main groups of generators: Makefile, Ninja, and Visual Studio.

It’s fair to say that because each group has a build system separate paradigm, they have separate code paths in CMake. This implies that all the Makefile generators – like NMake and Makefile – share some, or a lot of, common code. Same goes for the other groups.

Good Old Makefiles

Does it repro with the Makefile generator? A mingw installation later and:

> cmake -S. -B./build -G"Unix Makefiles" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DFETCHCONTENT_SOURCE_DIR_SDSLIB="../sdslib-cpp"
> cmake --build build
...
make[1]: *** No rule to make target '../_deps/sdslib-build/CMakeFiles/sdslib.dir/all', needed by 'CMakeFiles/rtek.dir/all'.  Stop.
make: *** [Makefile:91: all] Error 2

Yes! Looks like CMake’s Makefile generators are consistently failing. I’m almost certain this is a CMake Makefile generator bug2. It’s good to confirm that the Makefile generator works without FETCHCONTENT_SOURCE_DIR_SDSLIB set… and it does.

Diving In

Here’s what we know so far:

  • sdslib builds as a standalone project.
  • Everything works withoutFETCHCONTENT_SOURCE_DIR_SDSLIB set.
  • Visual Studio generator works with FETCHCONTENT_SOURCE_DIR_SDSLIB set.
  • NMake and Makefile generators don’t work with FETCHCONTENT_SOURCE_DIR_SDSLIB set.

Time to do something you are never supposed to do with CMake: dig into the generated files. I’ll use the Makefile generator since I’m familiar with Makefiles.

Because the CMake Makefile generator creates plain makefiles, we can just called them directly with debug information enabled.

> cd build
> make --debug=b
...
 File 'external/glad/CMakeFiles/glad.dir/build' does not exist.
Must remake target 'external/glad/CMakeFiles/glad.dir/build'.
Successfully remade target file 'external/glad/CMakeFiles/glad.dir/build'.
[ 11%] Built target glad
    Successfully remade target file 'external/glad/CMakeFiles/glad.dir/all'.
     File '../_deps/sdslib-build/CMakeFiles/sdslib.dir/all' does not exist.
    Must remake target '../_deps/sdslib-build/CMakeFiles/sdslib.dir/all'.
make[1]: *** No rule to make target '../_deps/sdslib-build/CMakeFiles/sdslib.dir/all', needed by 'CMakeFiles/rtek.dir/all'.  Stop.
make: *** [Makefile:91: all] Error 2

There’s some good hints here that confirm we are on the right path.

I have other libraries, like “glad”, that are built the same way as sdslib. We can see it’s able to build “glad” using the target *.dir/all. Remember these targets are generated by CMake, so they all follow the same procedure: 1) see that the *.dir/all file doesn’t exist, 2) try to build that target. Here we see more confirmation that it’s only sdslib that is effected when setting FETCHCONTENT_SOURCE_DIR_SDSLIB. Good.

Let’s grep for the missing target:

❯ grep -rn '../_deps/sdslib-build/CMakeFiles/sdslib.dir/all' build
build/CMakeFiles/Makefile2:111:C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/all: C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/CMakeFiles/sdslib.dir/all
build/CMakeFiles/Makefile2:186:CMakeFiles/rtek.dir/all: ../_deps/sdslib-build/CMakeFiles/sdslib.dir/all
build/CMakeFiles/Makefile2:319:C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/CMakeFiles/sdslib.dir/all:
build/CMakeFiles/Makefile2:323:.PHONY : C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/CMakeFiles/sdslib.dir/all
build/CMakeFiles/Makefile2:328: $(MAKE) $(MAKESILENT) -f CMakeFiles/Makefile2 C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/CMakeFiles/sdslib.dir/all

All the references are in build/CMakeFile/Makefile2.

This is the rule for making all of the rtek dependencies in Makefile2:

#=============================================================================
# Target rules for target CMakeFiles/rtek.dir

# All Build rule for target.
CMakeFiles\rtek.dir\all: ..\_deps\glfw-build\src\CMakeFiles\glfw.dir\all
CMakeFiles\rtek.dir\all: external\glad\CMakeFiles\glad.dir\all
CMakeFiles\rtek.dir\all: ..\_deps\spdlog-build\CMakeFiles\spdlog.dir\all
CMakeFiles\rtek.dir\all: ..\_deps\fmt-build\CMakeFiles\fmt.dir\all
CMakeFiles\rtek.dir\all: ..\_deps\sdslib-build\CMakeFiles\sdslib.dir\all
	$(MAKE) $(MAKESILENT) -f CMakeFiles\rtek.dir\build.make /nologo -$(MAKEFLAGS) CMakeFiles\rtek.dir\depend
	$(MAKE) $(MAKESILENT) -f CMakeFiles\rtek.dir\build.make /nologo -$(MAKEFLAGS) CMakeFiles\rtek.dir\build
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --progress-dir=C:\Users\stewa\source\repos\rtek\build\CMakeFiles --progress-num=22,23,24,25,26,27,28,29,30,31,32,33 "Built target rtek"
.PHONY : CMakeFiles\rtek.dir\all

There’s a reference to our ..\_deps\sdslib-build\CMakeFiles\sdslib.dir\all, our missing target. Good.

Except not good. The target can’t be found in the file. Nor can it be found in any of the other build files generated by CMake…

❯ grep -rni '..\_deps\sdslib-build\CMakeFiles\sdslib.dir\all' build
❯

Let’s see if we can find something that is a non-relative path, or at least contains part of the target path. Searching for _deps\sdslib-build\CMakeFiles\sdslib.dir\all in Makefile2 reveals a target that looks suspiciously close to the one we want:

#=============================================================================
# Target rules for target C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/CMakeFiles/sdslib.dir

# All Build rule for target.
C:\Users\stewa\source\repos\rtek\_deps\sdslib-build\CMakeFiles\sdslib.dir\all:
	$(MAKE) $(MAKESILENT) -f C:\Users\stewa\source\repos\rtek\_deps\sdslib-build\CMakeFiles\sdslib.dir\build.make /nologo -$(MAKEFLAGS) C:\Users\stewa\source\repos\rtek\_deps\sdslib-build\CMakeFiles\sdslib.dir\depend
	$(MAKE) $(MAKESILENT) -f C:\Users\stewa\source\repos\rtek\_deps\sdslib-build\CMakeFiles\sdslib.dir\build.make /nologo -$(MAKEFLAGS) C:\Users\stewa\source\repos\rtek\_deps\sdslib-build\CMakeFiles\sdslib.dir\build
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --progress-dir=C:\Users\stewa\source\repos\rtek\build\CMakeFiles --progress-num=34,35 "Built target sdslib"
.PHONY : C:\Users\stewa\source\repos\rtek\_deps\sdslib-build\CMakeFiles\sdslib.dir\all

Ok, that’s clearly the target that we are interested in. Except it’s using its absolute path, not the relative path like the target name referencing it.

Let’s do the unthinkable. Let’s edit the generated file target to match the expected target:

  • C:\Users\stewa\source\repos\rtek\_deps\sdslib-build\CMakeFiles\sdslib.dir\all -> ..\_deps\sdslib-build\CMakeFiles\sdslib.dir\all
[ 66%] Linking CXX static library sdslibd.lib
[ 66%] Built target sdslib
...
[100%] Linking CXX executable rtekd.exe
[100%] Built target rtek
make[1]: *** No rule to make target '../_deps/sdslib-build/all', needed by 'all'.  Stop.
make: *** [Makefile:91: all] Error 2

Boom! Got it! sdslib built!

Time For A Coffee

So there we have it. CMake is mislabeling the targets as absolute paths instead of relative paths.

Now we can see the next error No rule to make target '../_deps/sdslib-build/all'. How much do you want to bet it’s also a relative/absolute path mix up?

Let’s look for _deps/sdslib-build/all, see if it’s an absolute path as a target name, and the change it to the expected relative path ../_deps/sdslib-build/all.

#=============================================================================
# Directory level rules for directory C:/Users/stewa/source/repos/rtek/_deps/sdslib-build

# Recursive "all" directory target.
C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/all: C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/CMakeFiles/sdslib.dir/all
.PHONY : C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/all

What do you know. Note that this target is looking for the path that we converted from absolute to relative earlier. Let’s change that to a relative path too and any other references we can find. Find and replace C:/Users/stewa/source/repos/rtek/_deps/sdslib-build/all to ../_deps/sdslib-build/all.

…

And the project builds!! Beautiful.

Now we know cmake is mixing up relative and absolute paths to refer to the same target.

Cause Rooted!

So now what? Let’s try to find a workaround. Maybe passing a full path to FETCHCONTENT_SOURCE_DIR_SDSLIB:

cmake -S. -B./build -G"Unix Makefiles" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DFETCHCONTENT_SOURCE_DIR_SDSLIB="C:/Users/stewa/source/repos/sdslib-cpp"
...
make[1]: *** No rule to make target '../_deps/sdslib-build/CMakeFiles/sdslib.dir/all', needed by 'CMakeFiles/rtek.dir/all'.  Stop.
make: *** [Makefile:91: all] Error 2

Well, it was worth a shot. I’m tired of messing around this. Time to file a bug.

Bug report.

Post Bug Report High

How cool is that? Finding a bug in CMake, one of the most popular C++ build systems! It’s honesty time now where I admit I’m a CMake n00b. This is my first major project that uses it. So it felt pretty cool to be able to bug test it for them. And also very frustrating when I’m trying to learn it, but c’est-la-vie.

Sometimes programming is like that. You set out to solve one problem and you end up discovering a bug in CMake. Or your configs. Or poorly formatted code begging to be fixed. Or a cool thing that should probably wait but you’ll do anyway.

Sorry, what was I working on again?

Right, rtek.

If you made it this far and you’re interested in the source code, rtek is a game engine I’m (slowly) developing. sdslib is my develop-as-I-need-it C++ standard library. Feel free to reach out with questions, comments, or concerns.

Thanks for following along!

  1. Spoiler alert: Everything always goes south eventually. Plan for it. ↩

  2. At this point I’m 99.9% sure it’s CMake. But there’s always that voice in the back of my head saying expect the unexpected. Plus it’s CMake, hard to believe there would be this blatant of a bug. Right? ↩