To make it easier to accept contributions to zlib we should establish automated testing.

This commit uses the existing test/example.c testing. It connects those tests to AppVeyor, allowing AppVeyor to mark commits as passing or failing the tests. It also automatically runs tests when a pull request is submitted or updated.

There is some room for future expansion enabled, such as tracking the file and line of a failing test. This is thanks to @dankegel, who also got this working on CI providers other than AppVeyor.

In order for this to land properly in madler/zlib, the README.md (the renamed and reformatted README file) must be updated with new [appveyor-shield] and [appveyor-link] links. These can be obtained by @madler once they enable AppVeyor on madler/zlib.

When using configure, compiler is not correctly detected as gcc and -fPIC is not added to SFLAGS. This causes relocation issues when linking under Linux.

CVE-2018-25032 tracks a bug in zlib 1.2.11 which allows memory corruption when deflating (i.e., when compressing) if the input has many distant matches.

There is a fix from @madler at https://github.com/madler/zlib/commit/5c44459c3b28a9bd3283aaceab7c615f8020c531

@taviso reports at https://www.openwall.com/lists/oss-security/2022/03/24/1 that this patch never made it into a release, and at the time of writing no distros had picked it up as a fix.

Description:

• On each commit these workflows will be run to verify that project generation, source file compilation, and test cases run successfully.
• On each pull request a workflow will be run to verify that all fuzzing tests pass successfully.

Supports the following CI configurations:

• CMake
  • Ubuntu GCC
  • Ubuntu GCC OSB (out of source build)
  • Ubuntu Clang
  • Ubuntu Clang Debug
  • Windows MSVC Win32
  • Windows MSVC Win64
  • macOS Clang
  • macOS GCC
• Configure
  • Ubuntu GCC
  • Ubuntu GCC OSB (out of source build)
  • macOS GCC
• Fuzz
  • OSS-Fuzz on pull request only

Example runs can be found here: https://github.com/nmoinvaz/zlib/actions

This PR does not change any source code and only adds yaml workflows. Take from this PR what you need.

After the upgrade to macOS High Sierra several Java applications that compress data stopped working correctly. Example are KNIME Analytics Platform or Tomcat. I was able to track down the problem to a potential bug in the zlib version shipped with High Sierra (1.12.11). Version 1.2.5 which is used in Sierra works find. The bug is caused by changing the compression levels for different entries. CreateZip.java.txt (rename to .java) demonstrates the problem. Compile and start with /tmp 100 as arguments. It works fine on any operating system except macOS High Sierra. If you comment the three lines that are marked with "Triggers the bug" the program also works as expected. This is a blocker for any Java application that changes compression levels on macOS therefore please consider this a critical bug.

The checksum is calculated in the uncompressed PNG data and can be made much faster by using SIMD.

Tests in ARMv8 yielded an improvement of about 3x (e.g. walltime was 350ms x 125ms for 4096x4096 bytes executed 30 times). That results in at least 18% improvement in PNG image decoding in Chromium.

Further details at: https://bugs.chromium.org/p/chromium/issues/detail?id=688601

@madler: It is possible to release a new build with all build and CVE-2022-37434 fixes...? A 1.2.12.1 or 1.2.13?

It is really important to have a solution to current 1.2.12.

Tickets/PRs:

• https://github.com/madler/zlib/pull/337
• https://github.com/madler/zlib/issues/404
• https://github.com/madler/zlib/issues/604
• https://github.com/madler/zlib/pull/607
• https://github.com/madler/zlib/issues/608
• https://github.com/madler/zlib/issues/609
• https://github.com/madler/zlib/issues/613
• https://github.com/madler/zlib/pull/614
• https://github.com/madler/zlib/issues/615
• https://github.com/madler/zlib/issues/617
• https://github.com/madler/zlib/issues/618
• https://github.com/madler/zlib/issues/620 + https://github.com/Esri/zlib/commit/05b33dd27dbbfc9360d5c2430e228d7dc48f92f6
• https://github.com/madler/zlib/issues/622
• https://github.com/madler/zlib/issues/623
• https://github.com/madler/zlib/pull/624
• https://github.com/madler/zlib/issues/628
• https://github.com/madler/zlib/issues/629
• https://github.com/madler/zlib/issues/631
• https://github.com/madler/zlib/pull/632
• https://github.com/madler/zlib/issues/635
• https://github.com/madler/zlib/pull/638
• https://github.com/madler/zlib/pull/639
• https://github.com/madler/zlib/pull/644
• https://github.com/madler/zlib/pull/645
• https://github.com/madler/zlib/issues/646
• https://github.com/madler/zlib/issues/660
• https://github.com/madler/zlib/issues/661
• https://github.com/madler/zlib/pull/662
• https://github.com/madler/zlib/issues/668
• https://github.com/madler/zlib/issues/672
• https://github.com/madler/zlib/pull/677

In more:

• https://github.com/madler/zlib/pull/557
• https://github.com/madler/zlib/pull/657
• https://github.com/madler/zlib/pull/681
• https://github.com/madler/zlib/pull/691
• https://github.com/madler/zlib/pull/694

About GitHub:

• https://github.com/madler/zlib/pull/492
• https://github.com/madler/zlib/pull/506

Thanks in advance.

Linked to:

• https://github.com/madler/zlib/issues/422

cc: @gvollant.

Description of the problem is:

deflateInit () for raw deflate moves internal state to BUSY_STATE (where as it is INIT_STATE for zlib). BUSY_STATE is considered to be == INIT_STATE for raw deflate. So, when deflateEnd() is called, it has no way to identify whether stream is in the start or middle of core deflate operation() thus reports an Z_DATA_ERROR.

How does it affect:

//This will work: deflateInit2() deflate(strmp,Z_FINISH); deflateEnd()

//This will not work deflateInit2() while(cnt—) { deflate(strmp,Z_FINISH); deflateReset()<--required to re-initiate deflate() on next input. } deflateEnd();

test_def_init_term.c - a simple test case to reproduce issue.

On Windows, when calling gzopen on a plain text file, reading some bytes (at least 1), then calling gzseek(f, 0L, SEEK_SET), all subsequent gzread calls fail (they return 0).

This does not happen when opening a compressed stream.

In contrast, calling gzrewind(f) (which should be equivalent to the above, according to the documentation in "zlib.h") always works. Also, on Linux and MacOSX there is no such issue.

I'm using zlib 1.2.8, and Windows 7 32bit (but I believe 64bit is also affected).

Here is a simple test file which shows the problem (assuming "tst.tmp.txt" is any plain text file):

#include <stdio.h>
#include <zlib.h>

int main(int argc,char *argv[])
{
    char buf[1000];
    int ret;
    gzFile f = gzopen("tst.tmp.txt", "rb");
    //gzFile f = gzopen("tst.tmp.gz", "rb"); // this works

    while (!gzeof(f)) {
        ret = gzread(f, buf, 1);
        if (!ret) {
            printf("READ FAILED\n");
        } else {
            printf("%c %i\n", buf[0], buf[0]);
        }
    }
    printf("EOF\n");
    ret = gzseek(f, 0, SEEK_SET);
    printf("SEEK RETURNED: %i\n", ret);
    while (!gzeof(f)) {
        ret = gzread(f, buf, 1);
        if (!ret) {
            printf("READ FAILED\n");
        } else {
            printf("%c %i\n", buf[0], buf[0]);
        }
    }
    printf("EOF\n");

    return 0;
}

when running the code, the first while loop succeeds, then gzseek returns 0, and subsequent gzreads fail. If the first while loop is omitted, the rest succeeds, but as long as a single byte is read the bug is observed.

zlib has the ability to log runtime log messages. These log messages are only logged if a compile/build time flag is enabled ZLIB_DEBUG. This then means that it all depends on who/how the library is built for logging to be available at runtime.

Recently, we have been trying to solve an issue in zlib that's shipped in macos aarch64 systems. The zlib shipped in that system wasn't built with ZLIB_DEBUG enabled (we verified it by calling zlibCompileFlags() at runtime and checking its output as noted in https://github.com/madler/zlib/blob/master/zlib.h#L1185). This effectively means that to be able to investigate the issue we need to rebuild that library with ZLIB_DEBUG, which isn't feasible given the lack of access to the actual source and the actual build command/configurations that were used to build it originally.

Would it be feasible to introduce a way where this crucial logging can be enabled at runtime without having to rebuild the library? Perhaps an explicit API on the zlib library which enables the logging or maybe some runtime environment variable which enables it? I don't know if there's a precedent to allow environment variables to enable things like this, so if that's not a possibility then an API that the library's client is expected to call, should be fine perhaps?

Hi Mr. Adler,

I can't build zlibwapi.dll from your new version 1.2.12 because of errors during the compiling process using Visual Studio v17 2022. The previous version 1.2.11 worked. I see many people have this problem. Can you please make a new release?

Thank you so much.

All the other functions in the same file were converted from K&R style to normal function argument declarations in d004b047838a7e803818b4973a2e39e0ff8c1fa2

This does not work when enabling 64bit time_t with glibc which is enabled with -D_TIME_BITS=64, since it also needs _FILE_OFFSET_BITS=64 and this does not work when its undefined explicitly

Signed-off-by: Khem Raj [email protected]

Hi!

Please find my changes to fix the broken build for the good old Amiga below. It should not harm any other platform but just bring the Amiga relevant files up to date again. Besides that, I took the freedom to add a few lines of documentation to make zlib more accessible to other Amiga developers. I would really appreciate if my PR would be accepted.

Thanks for your efforts,

Christoph

I'm trying build minizip with a locally built zlib because my circumstances don't allow for an internet connection while building minizip. Minizip will basically only look for zlib with find_package or fetch it, so I've had to add package export to my local copy of zlib. But I'm not sure there's a downside to having that all the time, so maybe you add it to the CMakeLists.txt? Replace:

    install(TARGETS zlib zlibstatic
    	  EXPORT ${PROJECT_NAME}
        RUNTIME DESTINATION "${INSTALL_BIN_DIR}"
        ARCHIVE DESTINATION "${INSTALL_LIB_DIR}"
        LIBRARY DESTINATION "${INSTALL_LIB_DIR}" )

with

    install(TARGETS zlib zlibstatic
    	  EXPORT ${PROJECT_NAME}
        RUNTIME DESTINATION "${INSTALL_BIN_DIR}"
        ARCHIVE DESTINATION "${INSTALL_LIB_DIR}"
        LIBRARY DESTINATION "${INSTALL_LIB_DIR}" )

    install(EXPORT ${PROJECT_NAME}
            DESTINATION "${INSTALL_LIB_DIR}/cmake/${PROJECT_NAME}"
            NAMESPACE "ZLIB::")

Or something similar at any rate! Thanks!

I prefer to use FetchContent for dependency management. With this:

FetchContent_Declare(
    zlib
    GIT_REPOSITORY "https://github.com/madler/zlib.git"
    FIND_PACKAGE_ARGS NAMES ZLIB
)
FetchContent_MakeAvailable(zlib)

the whole process of cloning the repo, generating and building it as a dependency might be automated by cmake.

The problem is that the example executable relies on the assumption that the whole project with the headers is the top-level cmake project. And of course this is not true in case of FetchContent which will download zlib as a subdirectory.

There are several different approaches on how to fix this. But the simplest one I can see is to add an option to disable tests and examples, like ZLIB_BUILD_TESTS.

I can provide an example project that uses zlib as 3rdparty via FetchContent to reproduce the issue if needed.

Documentation says z_size_t should be defined as smallest integer type that can hold a pointer, but unsigned long is 32 bits on 64-bit Windows and as such can't hold a pointer and will truncate. Only integer type that doesn't require any header file and can contain a pointer is unsigned __int64. This will also affect total_in and total_out in z_stream_s; compress, compress2, compressBound, uncompress and uncompress2 that use unsigned long to hold length.

To detect 64-bit Windows, preprocessor macro _WIN64 can be used.