How much the libgit2 test failed? (How should I interpret the results?)

Question

I have fetched/merged the last changes from the libgit2 repository (the commit 4ae29053d5), created the build subdirectory, and compiled the library on Windows 7 using Visual Studio 2010 this way:

e:\Software\libgit2\build>cmake .. -G"Visual Studio 10" -DTHREADSAFE=ON -DSTDCALL=OFF
-- The C compiler identification is MSVC 16.0.40219.1
-- Check for working C compiler using: Visual Studio 10
-- Check for working C compiler using: Visual Studio 10 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
zlib was not found; using bundled 3rd-party sources.
-- Found PythonInterp: c:/Python26/python.exe (found version "2.6.5")
-- Configuring done
-- Generating done
-- Build files have been written to: E:/Software/libgit2/build

e:\Software\libgit2\build>cmake --build .

Microsoft (R) Visual Studio Version 10.0.40219.1.
Copyright (C) Microsoft Corp. All rights reserved.
1>------ Build started: Project: ZERO_CHECK, Configuration: Debug Win32 ------
1>  Checking Build System
1>  CMake does not need to re-run because E:/Software/libgit2/build/CMakeFiles/g
enerate.stamp is up-to-date.
2>------ Build started: Project: git2, Configuration: Debug Win32 ------
3>------ Build started: Project: libgit2_clar, Configuration: Debug Win32 ------

2>  Building Custom Rule E:/Software/libgit2/CMakeLists.txt
2>  CMake does not need to re-run because E:\Software\libgit2\build\CMakeFiles\g
enerate.stamp is up-to-date.
2>  precompiled.c
3>  Building Custom Rule E:/Software/libgit2/CMakeLists.txt
[...]
3>  renames.c
3>  submodules.c
3>  lookup.c
3>  status.c
3>  basic.c
3>  trace.c
3>     Creating library E:/Software/libgit2/build/Debug/libgit2_clar.lib and obj
ect E:/Software/libgit2/build/Debug/libgit2_clar.exp
3>     Creating library E:/Software/libgit2/build/Debug/libgit2_clar.lib and obj
ect E:/Software/libgit2/build/Debug/libgit2_clar.exp
3>  libgit2_clar.vcxproj -> E:\Software\libgit2\build\Debug\libgit2_clar.exe
4>------ Build started: Project: ALL_BUILD, Configuration: Debug Win32 ------
4>  Building Custom Rule E:/Software/libgit2/CMakeLists.txt
4>  CMake does not need to re-run because E:\Software\libgit2\build\CMakeFiles\g
enerate.stamp is up-to-date.
4>  Build all projects
========== Build: 4 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========    

Everything seems OK. When launching the test, it fails:

e:\Software\libgit2\build>ctest -V
UpdateCTestConfiguration  from :E:/Software/libgit2/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :E:/Software/libgit2/build/DartConfiguration.tcl
Test project E:/Software/libgit2/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: libgit2_clar

1: Test command: E:\Software\libgit2\build\Debug\libgit2_clar.exe "-ionline"
1: Test timeout computed to be: 9.99988e+006
1: Loaded 195 suites:
1: Started
1:
1: attr::file....
1: attr::flags...
1: attr::ignore.....
[...]
1: status::submodules......
1: status::worktree..................................
1: status::worktree::init........
1: submodule::lookup.....
1: submodule::modify....
1: submodule::status.......
1: threads::basic.
1: trace::trace......
1:
1:   1) Failure:
1: diff::workdir::submodules [..\tests-clar\diff\workdir.c:825]
1:   33 != exp.lines
1:   33 != 35
1:
1/1 Test #1: libgit2_clar .....................***Failed  250.12 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) = 250.20 sec

The following tests FAILED:
          1 - libgit2_clar (Failed)
Errors while running CTest

Is it a know situation? It seems that a lot of (sub)tests passed. I do not know the test framework. Does the messages at the end mean that only the diff::workdir::submodules failed?

Answer 1

I do not know the test framework.

Libgit2 relies on the Clar test framework.

Does the messages at the end mean that only the diff::workdir::submodules failed?

Yes. More precisely, it points you to the failing assertion

Is it a known situation?

No. The libgit2 team works hard to always have the tests passing.

I've tried locally and I cannot reproduce this issue. This gist contains the build and test run against libgit2@4ae29053.

This has been run on Win 7 (32 bits) platform against Visual Studio 2010 as well.

The only difference I see is that you're running an older version of Python, but that shouldn't be a problem. Clar relies on Python to generate the test suite (more info about this in the README file of the project) and supports Python back to version 2.5.

If you can successfully reproduce this failure, please open an issue in the libgit2 issue tracker as this may require a more deeper troubleshooting that may not fit to the StackOverflow format.