Hi, I needed to use expat in an STM32 application written in C. The project is compiled in Embitz in Windows for STM32 target. I tried to follow the example code of e.g. element.c within my porject but start and end element handler never get called. Although if I try to compile it with CMake for windows as separate application and give xml file as parameter to compile *.exe example it parses the xml file correctly. Do I have to have some special defines for embedded targets as STM32? Initially I was having assertion due Hash salt but then I set the hash salth using XML_SetHashSatl(parser, 1) which helped. But still the start and end element handler never get called when I run my project in embedded target.

Trying to upgrade to the latest version of expat using MSVC 2022, I find it impossible to link against expat, when compiled with the static runtime options. Here my cmake variables:

  set (EXPAT_SHARED_LIBS OFF)
  set (EXPAT_BUILD_EXAMPLES OFF)
  set (EXPAT_BUILD_TESTS OFF)
  set (EXPAT_BUILD_TOOLS OFF)
  set (EXPAT_BUILD_TOOLS OFF)
  if (WITH_STATIC_RUNTIME)
    #expat is overriding this, so enforce it to be
    set(EXPAT_MSVC_STATIC_CRT ON CACHE BOOL "Use /MT flag (static CRT) when compiling in MSVC" FORCE)
  endif()

Unfortunately, when trying to link against the library, i get an undefined symbol:

libexpat.lib(xmlparse.c.obj) : error LNK2019: unresolved external symbol __imp_rand_s referenced in function generate_hash_secret_salt

additionally there are warnings, that indicate that the static runtime seems to be ignored:

LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library
LINK : warning LNK4217: symbol '_errno' defined in 'libucrt.lib(errno.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function 'ENTROPY_DEBUG'
LINK : warning LNK4217: symbol '__acrt_iob_func' defined in 'libucrt.lib(_file.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function 'ENTROPY_DEBUG'
LINK : warning LNK4217: symbol '__stdio_common_vfprintf' defined in 'libucrt.lib(output.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function '_vfprintf_l'
LINK : warning LNK4217: symbol 'free' defined in 'libucrt.lib(free.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function 'parserCreate'
LINK : warning LNK4217: symbol 'malloc' defined in 'libucrt.lib(malloc.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function 'parserCreate'
LINK : warning LNK4217: symbol 'realloc' defined in 'libucrt.lib(realloc.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function 'parserCreate'
LINK : warning LNK4217: symbol 'strtoul' defined in 'libucrt.lib(strtox.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function 'ENTROPY_DEBUG'
LINK : warning LNK4217: symbol 'getenv' defined in 'libucrt.lib(getenv.obj)' is imported by 'libexpat.lib(xmlparse.c.obj)' in function 'ENTROPY_DEBUG'

Before we can drop GNU Autotools, we need at least:

• [x] (1) Get MinGW support working in CMake — #334
• [x] (2) Add support for make run-xmltest to CMake, including test execution using Wine — depends on (1) — #334
• [x] (3) Migrate coverage.sh to use CMake — depends on (1) and (2), #357
• [x] (4) Migrate qa.sh to use CMake — depends on (1) and (2)
• [x] (5) Replace all remaining Autotools bits in Travis CI by CMake — depends on (1) and (2), potentially also on (4)
• [ ] (6) Port all remaining needed bits from configure.ac / Makefile.am to CMake
• [x] (7) Drop ~or adapt~ expat.spec RPM upstream package — ccd1cbc14db64f81f7404c5da37e1a9b9fa6de3b
• [x] (8) Implement equivalent to make distcheck for CMake — depends on (2)
• [x] (9) Port distribute.sh to use CMake — depends on (8)
• [ ] (10) Compare expat_config.h before and after —in part addressed by #437
• [ ] (11) Check CMake variables names for namespace prefix and and visibility

If you want to help with some of these, please get in touch to discuss further details before you start. Thank you!

I'm working on a filter to convert hundreds and hundreds of files to XML. To test it, I want to convert them and quickly find out if there were any errors that generated bad XML. At the moment, xmlwf stops after encountering the first file with an error. The -k option instructs xmlwf to continue processing. It's been super helpful to my test setup.

Hi!

I would like to report an issue I am having while trying to update the Expat dependency in the Exiv2 project, from the version 2.2.5 to the version 2.2.6.

In Exiv2 we bring the expat dependency (and others) with conan. I actually wrote a conan recipe for expat which is maintained here: https://github.com/Pix4D/conan-expat

As you can see here, in this Exiv2 branch I am just trying to bump the expat version from 2.2.5 to 2.2.6: https://github.com/Exiv2/exiv2/compare/master...piponazo:expat226

And by just doing that change, one of our travis CI jobs started to fail with this error message:

[ 52%] Linking CXX executable ../bin/exiv2
../lib/libexiv2.so.0.27.0.2: undefined reference to `getrandom'

This is just happening for the Linux job (Ubuntu 14.04) with clang-5. The rest of jobs in the matrix compile the project correctly: https://travis-ci.org/piponazo/exiv2/builds/458033518

I saw that there are other issues related somehow with the getrandom function (for example #172). I would like to know if I need to add some definition (and which one) to solve this issue.

Thanks in advance! 😸

This PR proposes an enhancement for ability to parse multiple XMLs in single stream, file, dump or data buffer in one go (so called bulk-XML parsing).

This solution is part of similar RFE for tdom Tcl-module, which is using libexpat. Thus to avoid possible maintenance or merge issue, we'd like to see the related changes as part of libexpat.

The change is totally backwards compatible and behaves completely as before if the BulkXMLEnd-handler is not set (it'd still throw an error XML_ERROR_JUNK_AFTER_DOC_ELEMENT by further XML data in stream as libexpat always did it). If BulkXMLEnd-handler becomes set (with new function XML_SetBulkXMLEndHandler), the bulk-parsing will be enabled and parser is able to process multiple XMLs in single stream in one go. In this case the BulkXMLEnd-handler will be invoked after every fulfilled chunk XML if it was successfully parsed to the end of chunk. So if stream or file contains 100 XMLs, this handler will be called 100 times after every processed XML.

The test-cases of tdom (tcl-side) illustrating the behavior could be found here.

I'm really not familiar with the codebase, so please review carefully. libexpat is a dependency of a dependency of a dependency for me, and I'm just trying to get everything building with more warnings fixed.

The _GNU_SOURCE thing in particular could maybe be addressed differently. Perhaps it doesn't need to be defined at all?

Hi @hartwork,

The integration is ready for first review.

Also, you can go ahead and sign-up at https://app.fuzzit.dev so I'll be able to add you to the libexpat organization so you can receive emails if anything pops up (Until our Github integration is ready:))

CCing @catenacyber who worked on the PR

Is there a specific reason that the current implementation does not just use the published rand_s() API call (which uses RtlGenRandom) and instead rolls its own dynamic library load to call RtlGenRandom?

https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/rand-s?view=vs-2019

EDIT: I'd be happy to submit a patch, because I think it would simplify things quite a bit.

Starting with 2.2.1, libexpat added a LoadLibrary() call to load the ADVAPI32.DLL Windows system DLL to improve random numbers. This call however is prone to a known DLL hijacking vulnerability, with no (trivial) way to opt-out from this by apps making use of libexpat. The attack works by building a tailor-made ADVAPI32.DLL that exports the function required and called by libexpat, and copying that DLL to the directory of the user application or to the current directory.

My (already proposed in https://github.com/libexpat/libexpat/pull/62) patch is this:

--- expat.orig/xmlparse.c	2017-07-12 21:55:49.000000000 +0000
+++ expat/xmlparse.c	2017-07-14 19:46:29.000000000 +0000
@@ -777,6 +779,10 @@
 
 typedef BOOLEAN (APIENTRY *RTLGENRANDOM_FUNC)(PVOID, ULONG);
 
+#ifndef LOAD_LIBRARY_SEARCH_SYSTEM32
+# define LOAD_LIBRARY_SEARCH_SYSTEM32  0x00000800
+#endif
+
 /* Obtain entropy on Windows XP / Windows Server 2003 and later.
  * Hint on RtlGenRandom and the following article from libsodioum.
  *
@@ -786,7 +792,7 @@
 static int
 writeRandomBytes_RtlGenRandom(void * target, size_t count) {
   int success = 0;  /* full count bytes written? */
-  const HMODULE advapi32 = LoadLibrary(TEXT("ADVAPI32.DLL"));
+  const HMODULE advapi32 = LoadLibraryEx(TEXT("ADVAPI32.DLL"), NULL, LOAD_LIBRARY_SEARCH_SYSTEM32);
 
   if (advapi32) {
     const RTLGENRANDOM_FUNC RtlGenRandom

It will resolve the problem for Windows Vista and newer versions, thus covering all officially supported versions of Windows. For older versions the generally recommended method is to detect SYSTEM32 directory and prepend that to the loaded DLL name.

Hi Everyone,

I'm updating OpenSSH on Solaris 11.4 i86. libexpat is a dependency. libexpat 2.4.5 is having a little trouble:

$ make check -k
...
/bin/g++ -DHAVE_CONFIG_H -I. -I..   -DXML_ENABLE_VISIBILITY=1 -I./../lib -I/opt/ssh/incl
ude -DNDEBUG -Wall   -fexceptions -fno-strict-aliasing -g2 -O2 -m64 -march=native -fPIC
-pthread -MT runtestspp.o -MD -MP -MF .deps/runtestspp.Tpo -c -o runtestspp.o runtestspp
.cpp
In file included from runtestspp.cpp:36:0:
runtests.c: In function 'void test_billion_laughs_attack_protection_api()':
runtests.c:11712:32: error: 'NAN' was not declared in this scope
          parserWithoutParent, NAN)
                               ^
runtests.c:11734:32: error: 'INFINITY' was not declared in this scope
          parserWithoutParent, INFINITY)
                               ^
*** Error code 1

This patch appears to clear the issue. The self tests build and run without problems:

$ cat expat.patch 
--- tests/runtests.c
+++ tests/runtests.c 
@@ -60,6 +60,16 @@
 #  include <stdbool.h>
 #endif
 
+#if defined(__sun__)
+#  include <sunmath.h>
+#  if ! defined(INFINITY)
+#    define INFINITY (__builtin_huge_val())
+#  endif
+#  if ! defined(NAN)
+#    define NAN (__builtin_nan(""))
+#  endif
+#endif
+
 #include "expat.h"
 #include "chardata.h"
 #include "structdata.h"
@@ -6636,7 +6646,8 @@
   XML_Char **elemstr = (XML_Char **)userData;
   if (xcstrcmp(elemstr[0], name) != 0) {
     char buffer[1024];
-    sprintf(buffer, "unexpected end string: '%" XML_FMT_STR "'", name);
+    snprintf(buffer, sizeof(buffer), "unexpected end string: '%" XML_FMT_STR "'", name);
+    buffer[sizeof(buffer)-1] = '\0';  /* ensure NULL termination */
     fail(buffer);
   }
   triplet_end_flag = XML_TRUE;

On systems like Flatpak and Debian, the cmake file is installed under a subdirectory including an extra level for the compiler triplet (e.g. /usr/lib/x86_64-linux-gnu/cmake/expat-2.4.6/expat.cmake). This adds an extra hop in those cases so to cover them.

An alternative would be to generate it at build time (which is what we do in KDE), but I'm not sure this is really feasible considering that expat has two supported build systems and this approach covers them both.

Related to https://github.com/libexpat/libexpat/issues/501

In windows the name ELEMENT_TYPE is already used. Therefore there are problems when using libexpat in VisualStudio (winioctl.h). Changing the name of the struct ELEMENT_TYPE to XML_ELEMENT_TYPE in xmlparse.c would solve this problem.

Add CIFuzz workflow action to have fuzzers build and run on each PR.

This is a service offered by OSS-Fuzz where Expat already runs. CIFuzz can help detect catch regressions and fuzzing build issues early, and has a variety of features (see the URL above). In the current PR the fuzzers gets build on a pull request and will run for 300 seconds.

Signed-off-by: David Korczynski [email protected]

The existing Expat coverage tests all live in the file tests/runtests.c. This makes for a very large source file, which makes understanding the tests and refactoring for common behaviour very hard. This PR pulls tests/runtests.c apart into a number of separate source files grouping together similar tests, and similar support and handler functions. It does not change the code flow of any of the tests, so does not change the current test coverage.

Hopefully this will allow us to refactor the tests more easily in the future, for example removing a lot of the duplication in support functions.

/*
 * This fuzzer is generated by UTopia with some manual modifications.
 * (UTopia Project: https://github.com/Samsung/UTopia)
 */

#include <expat.h>
#include <fstream>
#include "FuzzedDataProvider.h"
#include <stdio.h>
#include <expat.h>

#ifdef XML_LARGE_SIZE

#  define XML_FMT_INT_MOD "ll"
#else
#  define XML_FMT_INT_MOD "l"
#endif

#ifdef XML_UNICODE_WCHAR_T
#  include <wchar.h>
#  define XML_FMT_STR "ls"
#else
#  define XML_FMT_STR "s"
#endif

bool
saveFile(std::string Path, std::string Content) {
  std::ofstream OFS(Path);
  if (! OFS.is_open())
    return false;

  OFS << Content;
  return true;
}

static void XMLCALL
startElement(void *userData, const XML_Char *name, const XML_Char **atts) {
  int i;
  int *depthPtr = (int *)userData;
  (void)atts;

  for (i = 0; i < *depthPtr; i++)
    putchar('\t');
  printf("%" XML_FMT_STR "\n", name);
  *depthPtr += 1;
}

static void XMLCALL
endElement(void *userData, const XML_Char *name) {
  int *depthPtr = (int *)userData;
  (void)name;

  *depthPtr -= 1;
}

void
fuzz_expat(FuzzedDataProvider &provider) {
  auto input = provider.ConsumeRemainingBytesAsString();
  const char *file_name = "input";
  saveFile(file_name, input);

  char buf[BUFSIZ];
  XML_Parser parser = XML_ParserCreate(NULL);
  int done;
  int depth = 0;

  XML_SetUserData(parser, &depth);
  XML_SetElementHandler(parser, startElement, endElement);

  FILE *fp = fopen(file_name, "r");
  if (! fp)
    return;
  do {
    size_t len = fread(buf, 1, sizeof(buf), fp);
    done = len < sizeof(buf);
    if (XML_Parse(parser, buf, (int)len, done) == XML_STATUS_ERROR) {
      XML_ErrorString(XML_GetErrorCode(parser));
      XML_GetCurrentLineNumber(parser);
      XML_ParserFree(parser);
      return;
    }
  } while (! done && ! feof(fp));
  XML_ParserFree(parser);
  fclose(fp);
  return;
}

extern "C" int
LLVMFuzzerTestOneInput(const uint8_t *data, uint32_t size) {
  FuzzedDataProvider provider(data, size);
  fuzz_expat(provider);
  return 0;
}

The existing fuzzers don't cover very much of the possible state, and they don't test the validity of any of the arguments passed to the callback functions, so I wrote a more thorough fuzzer.