Update Hyperscan to support building Windows DLL for release & debug.

Also added a convenience function for validation, suitable for calling from Dotnet.

Hello,

I have been developing an app with very simple function to compile a pattern and use the pattern to match which needs use Hyperscan library:

After compiling the hyperscan library with following options: -DCMAKE_BUILD_TYPE=debug -DBOOST_ROOT=${BOOST_ROOT} -DBUILD_SHARED_LIBS=1 -DCMAKE_INSTALL_PREFIX=${USR_LIB_PATH}.

The App runs as expected at my compiling machine[model name : Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz], and then I copied my app and libhs.so to another machine [model name : Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz](Same X86 64bit machine). however it crashed every time at pattern load function when I execute the app.

The compiling machine and running machine has difference cpuinfo, not sure it has relationship with this.

Could some one can help me out. Thanks.

Here are the crash info:

rgdb.sh core.simplegrep.14441 
file: compiled magic version [521] does not match with shared library magic version [524]
gdb ./simplegrep -c core.simplegrep.14441
GNU gdb (GDB) Amazon Linux (7.6.1-64.33.amzn1)
    Copyright (C) 2013 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "x86_64-amazon-linux-gnu".
    For bug reporting instructions, please see:
    <http://www.gnu.org/software/gdb/bugs/>...
    Reading symbols from /home/david/simplegrep...done.
    [New LWP 14441]
    Missing separate debuginfo for /usr/lib64/libstdc++.so.6
    Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/0a/90c35d3174805453ea67a785446d628e298b59.debug
    Missing separate debuginfo for /lib64/libgcc_s.so.1
    Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/00/fa2883fb47b1327397bbf167c52f51a723d013.debug
    Core was generated by `./simplegrep outside ss.list'.
    Program terminated with signal 4, Illegal instruction.
#0  set_range (to=111, from=111, this=0x7ffee99b7910) at /home/tidy/work/app/hyperscan/hyperscan/src/util/bitfield.h:154
    154 /home/tidy/work/app/hyperscan/hyperscan/src/util/bitfield.h: No such file or directory.
    Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.167.amzn1.x86_64
    (gdb) bt
#0  set_range (to=111, from=111, this=0x7ffee99b7910) at /home/tidy/work/app/hyperscan/hyperscan/src/util/bitfield.h:154
#1  setRange (to=111 'o', from=111 'o', this=0x7ffee99b7910) at /home/tidy/work/app/hyperscan/hyperscan/src/util/charreach.h:106
#2  CharReach (to=111 'o', from=111 'o', this=0x7ffee99b7910) at /home/tidy/work/app/hyperscan/hyperscan/src/util/charreach.h:62
#3  ue2::AsciiComponentClass::add (this=0x1c3aca0, c=111) at /home/tidy/work/app/hyperscan/hyperscan/src/parser/AsciiComponentClass.cpp:124
#4  0x00007fe4c5fddcb1 in ue2::getLiteralComponentClass ([email protected]=111 'o', nocase=<optimized out>)
        at /home/tidy/work/app/hyperscan/hyperscan/src/parser/ComponentClass.cpp:421
#5  0x00007fe4c5e0b1cc in ue2::addLiteral ([email protected]=0x1c3ac50, c=111 'o', mode=...)
            at /home/tidy/work/app/hyperscan/hyperscan/src/parser/Parser.rl:181
#6  0x00007fe4c5fe78ca in ue2::parse ([email protected]=0x7ffee99ba6ca "outside", globalMode=...)
                at /home/tidy/work/app/hyperscan/hyperscan/src/parser/Parser.rl:1823
#7  0x00007fe4c5e1b7b1 in ue2::ParsedExpression::ParsedExpression (this=0x7ffee99b8570, index_in=<optimized out>, expression=0x7ffee99ba6ca "outside", flags=2, 
            actionId=<optimized out>, ext=0x0) at /home/tidy/work/app/hyperscan/hyperscan/src/compiler/compiler.cpp:116
#8  0x00007fe4c5e1d0c4 in ue2::addExpression (ng=..., [email protected]=0, expression=0x7ffee99ba6ca "outside", flags=2, ext=0x0, id=0)
    at /home/tidy/work/app/hyperscan/hyperscan/src/compiler/compiler.cpp:236
#9  0x00007fe4c5e19305 in ue2::hs_compile_multi_int ([email protected]=0x7ffee99b8a98, [email protected]=0x7ffee99b8a94, [email protected]=0x7ffee99b8aac, 
            [email protected]=0x0, [email protected]=1, [email protected]=1, [email protected]=0x0, [email protected]=0x7ffee99b8bf0, [email protected]=0x7ffee99b8bf8, g=...)
    at /home/tidy/work/app/hyperscan/hyperscan/src/hs.cpp:228
#10 0x00007fe4c5e19a13 in hs_compile ([email protected]=0x7ffee99ba6ca "outside", [email protected]=2, [email protected]=1, [email protected]=0x0, 
            [email protected]=0x7ffee99b8bf0, [email protected]=0x7ffee99b8bf8) at /home/tidy/work/app/hyperscan/hyperscan/src/hs.cpp:285
#11 0x0000000000400d31 in main (argc=<optimized out>, argv=<optimized out>) at /home/tidy/exam/simplegrep.c:163
(gdb) q

Compiling machine cpuinfo: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 69 model name : Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz stepping : 1 microcode : 0x17 cpu MHz : 2300.092 cache size : 4096 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm ida arat epb pln pts dtherm fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid xsaveopt bogomips : 4600.18 clflush size : 64 cache_alignment : 64 address sizes : 42 bits physical, 48 bits virtual power management:

Running machine cpuinfo:

cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 62 model name : Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz stepping : 4 microcode : 0x415 cpu MHz : 2494.028 cache size : 25600 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt bugs : bogomips : 4988.05 clflush size : 64 cache_alignment : 64 address sizes : 46 bits physical, 48 bits virtual power management:

-Tidy

I have built the hyperscan as '.so' library, and using it by nginx.

and then I got segment fault error

Program received signal SIGSEGV, Segmentation fault.
0x00000000001202b6 in ?? ()
Missing separate debuginfos, use: debuginfo-install glibc-2.17-196.el7.x86_64 nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64 pcre-8.32-15.el7.x86_64 zlib-1.2.7-15.el7.x86_64
(gdb) bt
#0  0x00000000001202b6 in ?? ()
#1  0x00007ffff5fd14b5 in ?? ()
#2  0x00007fffffffd920 in ?? ()
#3  0x00007ffff7de75e8 in _dl_relocate_object () from /lib64/ld-linux-x86-64.so.2
Backtrace stopped: frame did not save the PC
(gdb) info register
rax            0x0	0
rbx            0x555555b9d8a0	93824998824096
rcx            0x0	0
rdx            0x78bfbfd	126614525
rsi            0x7ffff5fd1490	140737320391824
rdi            0x0	0
rbp            0x7fffffffd7e0	0x7fffffffd7e0
rsp            0x7fffffffd7d8	0x7fffffffd7d8
r8             0x555555b9d8a0	93824998824096
r9             0x555555b9d8a0	93824998824096
r10            0x7fffffffd640	140737488344640
r11            0x7fffffffd740	140737488344896
r12            0x7ffff5c45fc8	140737316675528
r13            0x7	7
r14            0x7ffff5b5d020	140737315721248
r15            0x7ffff690b598	140737330066840
rip            0x1202b6	0x1202b6
eflags         0x10246	[ PF ZF IF RF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0

# objdump -d libhs.so

  00000000001202b0 <[email protected]>:
  1202b0:   ff 25 8a eb ca 00       jmpq   *0xcaeb8a(%rip)        # dcee40 <_GLOBAL_OFFSET_TABLE_+0x5e40>
  1202b6:   68 c5 0b 00 00          pushq  $0xbc5
  1202bb:   e9 90 43 ff ff          jmpq   114650 <_init+0x20>

The latest release dos not longer build in my debian packaging environment: I am building with default flags in Debian Testing.

Here is a snipped of the errors that occur a lot during build:

/home/user/tmp/hyperscan-5.4.0/src/hwlm/hwlm.c:136: undefined reference to `avx2_memset'
/usr/bin/ld: ../lib/libhs.a(stream.c.o): in function `roseCountingMiracleScan':
/home/user/tmp/hyperscan-5.4.0/src/rose/counting_miracle.h:68: undefined reference to `avx2_memcpy'
/usr/bin/ld: ../lib/libhs.a(stream.c.o): in function `roseCountingMiracleScanShufti':
/home/user/tmp/hyperscan-5.4.0/src/rose/counting_miracle.h:118: undefined reference to `avx2_memcpy'
/usr/bin/ld: /home/user/tmp/hyperscan-5.4.0/src/rose/counting_miracle.h:118: undefined reference to `avx2_memcpy'
/usr/bin/ld: ../lib/libhs.a(stream.c.o): in function `roseCountingMiracleScan':
/home/user/tmp/hyperscan-5.4.0/src/rose/counting_miracle.h:68: undefined reference to `avx2_memcpy'
/usr/bin/ld: ../lib/libhs.a(program_runtime.c.o): in function `roseCountingMiracleScan':
/home/user/tmp/hyperscan-5.4.0/src/rose/counting_miracle.h:68: undefined reference to `avx2_memcpy'
/usr/bin/ld: ../lib/libhs.a(program_runtime.c.o):/home/user/tmp/hyperscan-5.4.0/src/rose/counting_miracle.h:118: more undefined references to `avx2_memcpy' follow
/usr/bin/ld: ../lib/libhs.a(database.c.o): in function `print_database_string':
/home/user/tmp/hyperscan-5.4.0/src/database.c:405: undefined reference to `avx2_snprintf'
/usr/bin/ld: ../lib/libhs.a(database.c.o): in function `hs_serialize_database':
/home/user/tmp/hyperscan-5.4.0/src/database.c:86: undefined reference to `avx2_memset'
/usr/bin/ld: /home/user/tmp/hyperscan-5.4.0/src/database.c:105: undefined reference to `avx2_memcpy'
/usr/bin/ld: ../lib/libhs.a(database.c.o): in function `hs_deserialize_database_at':
/home/user/tmp/hyperscan-5.4.0/src/database.c:227: undefined reference to `avx2_memset'
/usr/bin/ld: ../lib/libhs.a(database.c.o): in function `db_copy_bytecode':
/home/user/tmp/hyperscan-5.4.0/src/database.c:196: undefined reference to `avx2_memcpy'
/usr/bin/ld: ../lib/libhs.a(database.c.o): in function `hs_deserialize_database':
/home/user/tmp/hyperscan-5.4.0/src/database.c:275: undefined reference to `avx2_memset'
/usr/bin/ld: ../lib/libhs.a(database.c.o): in function `db_copy_bytecode':
/home/user/tmp/hyperscan-5.4.0/src/database.c:196: undefined reference to `avx2_memcpy'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough8_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:956: undefined reference to `core2_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough16_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:968: undefined reference to `core2_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough8_queueInitState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:1073: undefined reference to `core2_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `core2_nfaExecGough16_queueInitState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:1075: undefined reference to `core2_memset'
/usr/bin/ld: ../lib/libhs.a(mpv.c.o): in function `nfaExecMpv_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/mpv.c:886: undefined reference to `core2_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough8_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:956: undefined reference to `corei7_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough16_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:968: undefined reference to `corei7_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough8_queueInitState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:1073: undefined reference to `corei7_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `corei7_nfaExecGough16_queueInitState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:1075: undefined reference to `corei7_memset'
/usr/bin/ld: ../lib/libhs.a(mpv.c.o): in function `nfaExecMpv_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/mpv.c:886: undefined reference to `corei7_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough8_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:956: undefined reference to `avx2_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough16_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:968: undefined reference to `avx2_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `nfaExecGough8_queueInitState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:1073: undefined reference to `avx2_memset'
/usr/bin/ld: ../lib/libhs.a(gough.c.o): in function `avx2_nfaExecGough16_queueInitState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/gough.c:1075: undefined reference to `avx2_memset'
/usr/bin/ld: ../lib/libhs.a(mpv.c.o): in function `nfaExecMpv_initCompressedState':
/home/user/tmp/hyperscan-5.4.0/src/nfa/mpv.c:886: undefined reference to `avx2_memset'
collect2: error: ld returned 1 exit status

hyperscan version: 5.1.1&5.1.0&5.00 boost version: 1.57&1.70 pcre version: 8.41 Ragel version: 6.9 cmake version: 3.14.2 & 3.12.1

os version: CentOS 7.3.1611 gcc version: gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)

I still get this error using Relwithdebinfo mode.

gcc error:

../lib/libchimera.a(ch_compile.cpp.o): In function `ch::ch_compile_multi_int(char const* const*, unsigned int const*, unsigned int const*, unsigned int, unsigned int, unsigned long, unsigned long, hs_platform_info const*, ch_database**)':
ch_compile.cpp:(.text+0xbf1): undefined reference to `ue2::mmbit_size(unsigned int)'
ch_compile.cpp:(.text+0x1034): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x1050): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x1055): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x119a): undefined reference to `ue2::CompileError::CompileError(unsigned int, std::string const&)'
ch_compile.cpp:(.text+0x119f): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x11a4): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x12b8): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x12d4): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x12d9): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x1320): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x133c): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x1341): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x13e8): undefined reference to `ue2::CompileError::CompileError(unsigned int, std::string const&)'
ch_compile.cpp:(.text+0x13ed): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x13f2): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x1450): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x146c): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x1471): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x14ad): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x14c9): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x14ce): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x150a): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x1526): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x152b): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x1760): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x177c): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x1781): undefined reference to `typeinfo for ue2::CompileError'
ch_compile.cpp:(.text+0x17ce): undefined reference to `ue2::CompileError::CompileError(std::string const&)'
ch_compile.cpp:(.text+0x17ea): undefined reference to `ue2::CompileError::~CompileError()'
ch_compile.cpp:(.text+0x17ef): undefined reference to `typeinfo for ue2::CompileError'
../lib/libchimera.a(ch_compile.cpp.o):(.gcc_except_table+0x180): undefined reference to `typeinfo for ue2::CompileError'
../lib/libchimera.a(ch_compile.cpp.o):(.gcc_except_table+0x1d8): undefined reference to `typeinfo for ue2::CompileError'
../lib/libchimera.a(ch_compile.cpp.o):(.gcc_except_table+0x230): undefined reference to `typeinfo for ue2::CompileError'
../lib/libchimera.a(ch_runtime.c.o): In function `multiCallback':
ch_runtime.c:(.text+0x6df): undefined reference to `mmbit_maxlevel_direct_lut'
ch_runtime.c:(.text+0x6e9): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x728): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x766): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x79c): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x7d6): undefined reference to `mmbit_root_offset_from_level'
../lib/libchimera.a(ch_runtime.c.o):ch_runtime.c:(.text+0x808): more undefined references to `mmbit_root_offset_from_level' follow
../lib/libchimera.a(ch_runtime.c.o): In function `multiCallback':
ch_runtime.c:(.text+0x88b): undefined reference to `mmbit_keyshift_lut'
ch_runtime.c:(.text+0x972): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x98c): undefined reference to `mmbit_keyshift_lut'
ch_runtime.c:(.text+0xa35): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0xa43): undefined reference to `mmbit_maxlevel_direct_lut'
ch_runtime.c:(.text+0xab0): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0xaf0): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0xb31): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0xb73): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0xbb4): undefined reference to `mmbit_root_offset_from_level'
../lib/libchimera.a(ch_runtime.c.o):ch_runtime.c:(.text+0xbf5): more undefined references to `mmbit_root_offset_from_level' follow
../lib/libchimera.a(ch_runtime.c.o): In function `ch_scan':
ch_runtime.c:(.text+0x13a2): undefined reference to `mmbit_maxlevel_direct_lut'
ch_runtime.c:(.text+0x1420): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x146e): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x14ac): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x14ec): undefined reference to `mmbit_root_offset_from_level'
ch_runtime.c:(.text+0x152a): undefined reference to `mmbit_root_offset_from_level'
../lib/libchimera.a(ch_runtime.c.o):ch_runtime.c:(.text+0x1568): more undefined references to `mmbit_root_offset_from_level' follow
collect2: error: ld returned 1 exit status
make[3]: *** [bin/unit-chimera] Error 1
make[2]: *** [unit/CMakeFiles/unit-chimera.dir/all] Error 2
make[1]: *** [unit/CMakeFiles/unit-chimera.dir/rule] Error 2
make: *** [unit-chimera] Error 2

this happens while linking unit-chimera

I get a compilation error when compiling Hyperscan 5.0.0 using the latest boost 1.68.0 libraries. I do not get this error when using the 1.67.0 boost libraries (I am able to sucessfully compile Hyperscan 5.0.0 using the older 1.67.0 boost libraries). I am compiling on Ubuntu 18 x64, with gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3).

Error message is: in limex_compile.cpp, generated: error: call of overloaded ‘distance(...)’ is ambiguous.

Steps to re-create the issue: Get boost libraries: cd ~/snort_src wget https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.gz tar -xvzf boost_1_68_0.tar.gz

Get hyperscan wget https://github.com/intel/hyperscan/archive/v5.0.0.tar.gz tar -xvzf v5.0.0.tar.gz mkdir hyperscan-5.0.0-build cd hyperscan-5.0.0-build/ cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DBOOST_ROOT=~/snort_src/boost_1_68_0/ ../hyperscan-5.0.0 make

Results from 'make' command: [ 1%] Generating src/parser/control_verbs.cpp [ 1%] Generating src/parser/Parser.cpp Scanning dependencies of target hs_compile [ 1%] Building CXX object CMakeFiles/hs_compile.dir/src/grey.cpp.o [ 1%] Building CXX object CMakeFiles/hs_compile.dir/src/hs.cpp.o [ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/compiler/asserts.cpp.o [ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/compiler/compiler.cpp.o [ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/compiler/error.cpp.o [ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/engine_description.cpp.o [ 3%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_compile.cpp.o [ 3%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_compile_util.cpp.o [ 3%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_confirm_compile.cpp.o [ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_engine_description.cpp.o [ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/flood_compile.cpp.o [ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/teddy_compile.cpp.o [ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/teddy_engine_description.cpp.o [ 5%] Building CXX object CMakeFiles/hs_compile.dir/src/hwlm/hwlm_build.cpp.o [ 5%] Building CXX object CMakeFiles/hs_compile.dir/src/hwlm/hwlm_literal.cpp.o [ 5%] Building CXX object CMakeFiles/hs_compile.dir/src/hwlm/noodle_build.cpp.o [ 6%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/accel_dfa_build_strat.cpp.o [ 6%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/accelcompile.cpp.o [ 6%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/castlecompile.cpp.o [ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/dfa_build_strat.cpp.o [ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/dfa_min.cpp.o [ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/goughcompile.cpp.o [ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/goughcompile_accel.cpp.o [ 8%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/goughcompile_reg.cpp.o [ 8%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/mcclellancompile.cpp.o [ 8%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/mcclellancompile_util.cpp.o [ 9%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/mcsheng_compile.cpp.o [ 9%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/limex_compile.cpp.o /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp: In function ‘u32 ue2::{anonymous}::addSquashMask(const ue2::{anonymous}::build_info&, const NFAVertex&, std::vector<boost::dynamic_bitset<> >&)’: /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp:983:54: error: call of overloaded ‘distance(std::vector<boost::dynamic_bitset<> >::iterator, __gnu_cxx::__normal_iterator<boost::dynamic_bitset<>*, std::vector<boost::dynamic_bitset<> > >&)’ is ambiguous return verify_u32(distance(squash.begin(), it)); ^ In file included from /usr/include/c++/7/bits/stl_algobase.h:66:0, from /usr/include/c++/7/bits/char_traits.h:39, from /usr/include/c++/7/string:40, from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/nfa_kind.h:40, from /home/noah/snort_src/hyperscan-5.0.0/src/nfagraph/ng_holder.h:41, from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.h:37, from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp:34: /usr/include/c++/7/bits/stl_iterator_base_funcs.h:138:5: note: candidate: typename std::iterator_traits<_Iterator>::difference_type std::distance(_InputIterator, _InputIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<boost::dynamic_bitset<>*, std::vector<boost::dynamic_bitset<> > >; typename std::iterator_traits<_Iterator>::difference_type = long int] distance(_InputIterator __first, _InputIterator __last) ^~~~~~~~ In file included from /home/noah/snort_src/boost_1_68_0/boost/range/distance.hpp:18:0, from /home/noah/snort_src/boost_1_68_0/boost/range/functions.hpp:21, from /home/noah/snort_src/boost_1_68_0/boost/range/iterator_range_core.hpp:38, from /home/noah/snort_src/boost_1_68_0/boost/range/iterator_range.hpp:13, from /home/noah/snort_src/hyperscan-5.0.0/src/util/graph_range.h:54, from /home/noah/snort_src/hyperscan-5.0.0/src/util/ue2_graph.h:33, from /home/noah/snort_src/hyperscan-5.0.0/src/nfagraph/ng_holder.h:44, from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.h:37, from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp:34: /home/noah/snort_src/boost_1_68_0/boost/iterator/distance.hpp:49:9: note: candidate: typename boost::iterators::iterator_difference::type boost::iterators::distance_adl_barrier::distance(SinglePassIterator, SinglePassIterator) [with SinglePassIterator = __gnu_cxx::__normal_iterator<boost::dynamic_bitset<>*, std::vector<boost::dynamic_bitset<> > >; typename boost::iterators::iterator_difference::type = long int] distance(SinglePassIterator first, SinglePassIterator last) ^~~~~~~~ CMakeFiles/hs_compile.dir/build.make:720: recipe for target 'CMakeFiles/hs_compile.dir/src/nfa/limex_compile.cpp.o' failed make[2]: *** [CMakeFiles/hs_compile.dir/src/nfa/limex_compile.cpp.o] Error 1 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/hs_compile.dir/all' failed make[1]: *** [CMakeFiles/hs_compile.dir/all] Error 2 Makefile:129: recipe for target 'all' failed make: *** [all] Error 2

Results from cmake ( works OK, no errors): DBOOST_ROOT=~/snort_src/boost_1_68_0/ ../hyperscan-5.0.0 -- The C compiler identification is GNU 7.3.0 -- The CXX compiler identification is GNU 7.3.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test ARCH_64_BIT -- Performing Test ARCH_64_BIT - Success -- Performing Test ARCH_32_BIT -- Performing Test ARCH_32_BIT - Failed -- Default build type 'Release with debug info' -- Boost version: 1.68.0 -- Found PythonInterp: /usr/bin/python (found version "2.7.15") -- Build date: 2018-09-03 -- gcc version 7.3.0 -- g++ version 7.3.0 -- Looking for include file unistd.h -- Looking for include file unistd.h - found -- Looking for include file intrin.h -- Looking for include file intrin.h - not found -- Looking for C++ include intrin.h -- Looking for C++ include intrin.h - not found -- Looking for include file x86intrin.h -- Looking for include file x86intrin.h - found -- Looking for C++ include x86intrin.h -- Looking for C++ include x86intrin.h - found -- Looking for posix_memalign -- Looking for posix_memalign - found -- Looking for _aligned_malloc -- Looking for _aligned_malloc - not found -- Performing Test HAS_C_HIDDEN -- Performing Test HAS_C_HIDDEN - Success -- Performing Test HAS_CXX_HIDDEN -- Performing Test HAS_CXX_HIDDEN - Success -- Looking for _LIBCPP_VERSION -- Looking for _LIBCPP_VERSION - not found -- generator is Unix Makefiles -- Performing Test HAS_C_ATTR_IFUNC -- Performing Test HAS_C_ATTR_IFUNC - Success -- Performing Test HAVE_SSSE3 -- Performing Test HAVE_SSSE3 - Success -- Performing Test HAVE_AVX2 -- Performing Test HAVE_AVX2 - Success -- Performing Test HAVE_AVX512 -- Performing Test HAVE_AVX512 - Failed -- Performing Test HAVE_CC_BUILTIN_ASSUME_ALIGNED -- Performing Test HAVE_CC_BUILTIN_ASSUME_ALIGNED - Success -- Performing Test HAVE_CXX_BUILTIN_ASSUME_ALIGNED -- Performing Test HAVE_CXX_BUILTIN_ASSUME_ALIGNED - Success -- Performing Test HAVE__BUILTIN_CONSTANT_P -- Performing Test HAVE__BUILTIN_CONSTANT_P - Success -- Performing Test C_FLAG_Wvla -- Performing Test C_FLAG_Wvla - Success -- Performing Test C_FLAG_Wpointer_arith -- Performing Test C_FLAG_Wpointer_arith - Success -- Performing Test C_FLAG_Wstrict_prototypes -- Performing Test C_FLAG_Wstrict_prototypes - Success -- Performing Test C_FLAG_Wmissing_prototypes -- Performing Test C_FLAG_Wmissing_prototypes - Success -- Performing Test CXX_FLAG_Wvla -- Performing Test CXX_FLAG_Wvla - Success -- Performing Test CXX_FLAG_Wpointer_arith -- Performing Test CXX_FLAG_Wpointer_arith - Success -- Performing Test CC_SELF_ASSIGN -- Performing Test CC_SELF_ASSIGN - Failed -- Performing Test CXX_SELF_ASSIGN -- Performing Test CXX_SELF_ASSIGN - Failed -- Performing Test CC_PAREN_EQUALITY -- Performing Test CC_PAREN_EQUALITY - Failed -- Performing Test CXX_UNUSED_CONST_VAR -- Performing Test CXX_UNUSED_CONST_VAR - Success -- Performing Test CXX_IGNORED_ATTR -- Performing Test CXX_IGNORED_ATTR - Success -- Performing Test CXX_WEAK_VTABLES -- Performing Test CXX_WEAK_VTABLES - Failed -- Performing Test CXX_MISSING_DECLARATIONS -- Performing Test CXX_MISSING_DECLARATIONS - Success -- Performing Test CXX_UNUSED_LOCAL_TYPEDEFS -- Performing Test CXX_UNUSED_LOCAL_TYPEDEFS - Success -- Performing Test CXX_WUNUSED_VARIABLE -- Performing Test CXX_WUNUSED_VARIABLE - Success -- Building runtime for multiple microarchitectures -- Looking for mmap -- Looking for mmap - found -- Doxygen not found, unable to generate API reference -- Sphinx not found, unable to generate developer reference -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1") -- Checking for module 'libpcre=8.41' -- Requested 'libpcre = 8.41' but version of libpcre is 8.39 -- PCRE version 8.41 not found -- PCRE 8.41 not found -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
-- Checking for module 'sqlite3' -- Found sqlite3, version 3.22.0 -- Performing Test SQLITE_VERSION_OK -- Performing Test SQLITE_VERSION_OK - Success -- Looking for sqlite3_open_v2 -- Looking for sqlite3_open_v2 - found -- Looking for C++ include pthread_np.h -- Looking for C++ include pthread_np.h - not found -- Looking for pthread_setaffinity_np -- Looking for pthread_setaffinity_np - found -- Looking for malloc_info -- Looking for malloc_info - found -- Looking for shmget -- Looking for shmget - found -- PCRE 8.41 not found, not building hscollider -- Configuring done -- Generating done -- Build files have been written to: /home/noah/snort_src/hyperscan-5.0.0

Hi,

I'm currently trying to compile Hyperscan 4.70 on Slackware64-14.2 and i'm noticing this strange contradiction:

-- Checking for module 'libpcre=8.41' -- Found libpcre, version 8.41 -- PCRE version 8.41 -- PCRE 8.41 not found, not building hscollider -- Configuring done -- Generating done

This seems contradictory to me and is preventing me from building the hs-collider module.

GCC Version 5.5.0 G++ Version 5.5.0

Please let me know if you need any other information .

I compile the pattern of [\u4E00-\u9FA5] with default flags, when i using chines period（。）to match the pattern，the result is matched.According to the actual situation,the resule must be not matched.

When running the test suite (v5.1.1) under valgrind I'm getting the something similar to the following for a number of tests, including HyperscanArgChecks.ScratchSizeNoSize, HyperscanTestBehaviour.ScanSeveralGigabytesNoMatch, HyperscanTestBehaviour.CloseStreamMatch, HyperscanTestBehaviour.NoMainCB, HyperscanTestBehaviour.CloseStreamNoMatch, HyperscanTestBehaviour.CloseStreamAfterTermination, HyperscanTestBehaviour.VectoredX, HyperscanTestBehaviour.MultiStreamX, and many more.

Currently debugging.

==4607== Conditional jump or move depends on uninitialised value(s)
==4607==    at 0x3D60B3: hs_alloc_scratch (scratch.c:256)
==4607==    by 0x1D7325: (anonymous namespace)::HyperscanArgChecks_ScratchSizeNoSize_Test::TestBody() (arg_checks.cpp:2009)
==4607==    by 0x1B85DC: HandleSehExceptionsInMethodIfSupported<testing::Test, void> (gtest-all.cc:3562)
==4607==    by 0x1B85DC: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (gtest-all.cc:3598)
==4607==    by 0x1AB034: testing::Test::Run() [clone .part.451] (gtest-all.cc:3634)
==4607==    by 0x1AB328: Run (gtest-all.cc:3626)
==4607==    by 0x1AB328: testing::TestInfo::Run() [clone .part.452] (gtest-all.cc:3810)
==4607==    by 0x1AB4D0: Run (gtest-all.cc:3785)
==4607==    by 0x1AB4D0: testing::TestCase::Run() [clone .part.453] (gtest-all.cc:3928)
==4607==    by 0x1B2242: Run (gtest-all.cc:3914)
==4607==    by 0x1B2242: testing::internal::UnitTestImpl::RunAllTests() (gtest-all.cc:5799)
==4607==    by 0x1B8ABC: HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (gtest-all.cc:3562)
==4607==    by 0x1B8ABC: bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) (gtest-all.cc:3598)
==4607==    by 0x1AB689: testing::UnitTest::Run() (gtest-all.cc:5410)
==4607==    by 0x197D10: RUN_ALL_TESTS (gtest.h:20058)
==4607==    by 0x197D10: main (main.cpp:35)
==4607==  Uninitialised value was created by a heap allocation
==4607==    at 0x4837DBF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4607==    by 0x607707: allocate (new_allocator.h:111)
==4607==    by 0x607707: allocate (alloc_traits.h:436)
==4607==    by 0x607707: _M_allocate (stl_vector.h:296)
==4607==    by 0x607707: void std::vector<scatter_unit_u8, std::allocator<scatter_unit_u8> >::_M_realloc_insert<scatter_unit_u8 const&>(__gnu_cxx::__normal_iterator<scatter_unit_u8*, std::vector<scatter_unit_u8, std::allocator<scatter_unit_u8> > >, scatter_unit_u8 const&) (vector.tcc:427)
==4607==    by 0x606E6E: push_back (stl_vector.h:1085)
==4607==    by 0x606E6E: void ue2::add_scatter<scatter_unit_u8>(std::vector<scatter_unit_u8, std::allocator<scatter_unit_u8> >*, unsigned int, unsigned long long) (multibit_build.cpp:199)
==4607==    by 0x60725B: ue2::mmbBuildInitRangePlan(unsigned int, unsigned int, unsigned int, ue2::scatter_plan_raw*) [clone .part.193] (multibit_build.cpp:232)
==4607==    by 0x8A7671: ue2::buildStateScatterPlan(unsigned int, unsigned int, unsigned int, unsigned int, RoseStateOffsets const&, bool, unsigned int, unsigned int, unsigned int) (rose_build_scatter.cpp:81)
==4607==    by 0x86B3A1: ue2::RoseBuildImpl::buildFinalEngine(unsigned int) (rose_build_bytecode.cpp:3799)
==4607==    by 0x5B669F: ue2::RoseBuildImpl::buildRose(unsigned int) (rose_build_compile.cpp:1796)
==4607==    by 0x2A3B98: generateRoseEngine (compiler.cpp:352)
==4607==    by 0x2A3B98: ue2::build(ue2::NG&, unsigned int*) (compiler.cpp:422)
==4607==    by 0x2A10D2: ue2::hs_compile_multi_int(char const* const*, unsigned int const*, unsigned int const*, hs_expr_ext const* const*, unsigned int, unsigned int, hs_platform_info const*, hs_database**, hs_compile_error**, ue2::Grey const&) (hs.cpp:254)
==4607==    by 0x2A1AE9: hs_compile (hs.cpp:301)
==4607==    by 0x1D7297: (anonymous namespace)::HyperscanArgChecks_ScratchSizeNoSize_Test::TestBody() (arg_checks.cpp:2004)
==4607==    by 0x1B85DC: HandleSehExceptionsInMethodIfSupported<testing::Test, void> (gtest-all.cc:3562)
==4607==    by 0x1B85DC: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (gtest-all.cc:3598)

After upgrading Boost in homebrew/core from 1.61 to 1.62, hyperscan now fails to build.

The error is

[ 39%] Building CXX object CMakeFiles/hs.dir/src/nfagraph/ng_dominators.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++    -I/tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/build -I/tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1 -I/tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src -isystem /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/include -isystem /usr/local/include  -Os -w -pipe -march=native -mmacosx-version-min=10.11  -DNDEBUG -std=c++11 -Wall -Wextra -Wshadow -Wswitch -Wreturn-type -Wcast-qual -Wno-deprecated -Wnon-virtual-dtor -fno-strict-aliasing -fno-omit-frame-pointer -fvisibility=hidden -Wvla -Wpointer-arith -Wno-self-assign -Wno-unused-const-variable -Wno-ignored-attributes -Wweak-vtables -Wmissing-declarations -O2 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -mmacosx-version-min=10.11   -o CMakeFiles/hs.dir/src/nfagraph/ng_dominators.cpp.o -c /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.cpp
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.cpp:34:
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.h:38:
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_holder.h:32:
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_graph.h:46:
In file included from /usr/local/include/boost/graph/adjacency_list.hpp:246:
In file included from /usr/local/include/boost/graph/detail/adjacency_list.hpp:32:
/usr/local/include/boost/graph/graph_concepts.hpp:131:13: error: no matching function for call to 'degree'
        n = degree(v, g);
            ^~~~~~
/usr/local/include/boost/concept/usage.hpp:16:43: note: in instantiation of member function 'boost::concepts::BidirectionalGraph<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> >::~BidirectionalGraph' requested here
    ~usage_requirements() { ((Model*)0)->~Model(); }
                                          ^
/usr/local/include/boost/concept/detail/general.hpp:39:42: note: in instantiation of member function 'boost::concepts::usage_requirements<boost::concepts::BidirectionalGraph<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> > >::~usage_requirements' requested here
    static void failed() { ((Model*)0)->~Model(); }
                                         ^
/usr/local/include/boost/graph/graph_concepts.hpp:122:9: note: in instantiation of member function 'boost::concepts::requirement<boost::concepts::failed ************boost::concepts::usage_requirements<boost::concepts::BidirectionalGraph<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> > >::************>::failed' requested here
        BOOST_CONCEPT_USAGE(BidirectionalGraph) {
        ^
/usr/local/include/boost/concept/usage.hpp:29:7: note: expanded from macro 'BOOST_CONCEPT_USAGE'
      BOOST_CONCEPT_ASSERT((boost::concepts::usage_requirements<model>)); \
      ^
/usr/local/include/boost/concept/assert.hpp:43:5: note: expanded from macro 'BOOST_CONCEPT_ASSERT'
    BOOST_CONCEPT_ASSERT_FN(void(*)ModelInParens)
    ^
/usr/local/include/boost/concept/detail/general.hpp:71:51: note: expanded from macro 'BOOST_CONCEPT_ASSERT_FN'
    &::boost::concepts::requirement_<ModelFnPtr>::failed>    \
                                                  ^
/tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.cpp:67:16: note: in instantiation of function template specialization 'boost_ue2::lengauer_tarjan_dominator_tree<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &>, boost::adj_list_vertex_property_map<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, unsigned int, const unsigned int &, unsigned int ue2::NFAGraphVertexProps::*>, boost::iterator_property_map<std::__1::__wrap_iter<unsigned long *>, boost::adj_list_vertex_property_map<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, unsigned int, const unsigned int &, unsigned int ue2::NFAGraphVertexProps::*>, unsigned long, unsigned long &>, boost::iterator_property_map<std::__1::__wrap_iter<void **>, boost::adj_list_vertex_property_map<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, unsigned int, const unsigned int &, unsigned int ue2::NFAGraphVertexProps::*>, void *, void *&>, std::__1::vector<void *, std::__1::allocator<void *> >, boost::associative_property_map<boost::unordered::unordered_map<void *, void *, boost::hash<void *>, std::__1::equal_to<void *>, std::__1::allocator<std::__1::pair<void *const, void *> > > > >' requested here
    boost_ue2::lengauer_tarjan_dominator_tree(g, source, index_map, dfnum_map,
               ^
/tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.cpp:81:12: note: in instantiation of function template specialization 'ue2::calcDominators<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> >' requested here
    return calcDominators(boost::reverse_graph<NFAGraph, const NFAGraph &>(g.g),
           ^
/usr/local/include/boost/graph/detail/adjacency_list.hpp:1087:5: note: candidate template ignored: could not match 'undirected_graph_helper' against 'reverse_graph'
    degree(typename Config::vertex_descriptor u,
    ^
/usr/local/include/boost/graph/detail/adjacency_list.hpp:1514:5: note: candidate template ignored: could not match 'bidirectional_graph_helper_with_property' against 'reverse_graph'
    degree(typename Config::vertex_descriptor u,
    ^
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.cpp:34:
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.h:38:
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_holder.h:32:
In file included from /tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_graph.h:46:
In file included from /usr/local/include/boost/graph/adjacency_list.hpp:246:
In file included from /usr/local/include/boost/graph/detail/adjacency_list.hpp:32:
/usr/local/include/boost/graph/graph_concepts.hpp:138:13: error: no matching function for call to 'degree'
        n = degree(v, cg);
            ^~~~~~
/usr/local/include/boost/graph/graph_concepts.hpp:133:9: note: in instantiation of member function 'boost::concepts::BidirectionalGraph<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> >::const_constraints' requested here
        const_constraints(g);
        ^
/usr/local/include/boost/concept/usage.hpp:16:43: note: in instantiation of member function 'boost::concepts::BidirectionalGraph<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> >::~BidirectionalGraph' requested here
    ~usage_requirements() { ((Model*)0)->~Model(); }
                                          ^
/usr/local/include/boost/concept/detail/general.hpp:39:42: note: in instantiation of member function 'boost::concepts::usage_requirements<boost::concepts::BidirectionalGraph<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> > >::~usage_requirements' requested here
    static void failed() { ((Model*)0)->~Model(); }
                                         ^
/usr/local/include/boost/graph/graph_concepts.hpp:122:9: note: in instantiation of member function 'boost::concepts::requirement<boost::concepts::failed ************boost::concepts::usage_requirements<boost::concepts::BidirectionalGraph<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> > >::************>::failed' requested here
        BOOST_CONCEPT_USAGE(BidirectionalGraph) {
        ^
/usr/local/include/boost/concept/usage.hpp:29:7: note: expanded from macro 'BOOST_CONCEPT_USAGE'
      BOOST_CONCEPT_ASSERT((boost::concepts::usage_requirements<model>)); \
      ^
/usr/local/include/boost/concept/assert.hpp:43:5: note: expanded from macro 'BOOST_CONCEPT_ASSERT'
    BOOST_CONCEPT_ASSERT_FN(void(*)ModelInParens)
    ^
/usr/local/include/boost/concept/detail/general.hpp:71:51: note: expanded from macro 'BOOST_CONCEPT_ASSERT_FN'
    &::boost::concepts::requirement_<ModelFnPtr>::failed>    \
                                                  ^
/tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.cpp:67:16: note: in instantiation of function template specialization 'boost_ue2::lengauer_tarjan_dominator_tree<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &>, boost::adj_list_vertex_property_map<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, unsigned int, const unsigned int &, unsigned int ue2::NFAGraphVertexProps::*>, boost::iterator_property_map<std::__1::__wrap_iter<unsigned long *>, boost::adj_list_vertex_property_map<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, unsigned int, const unsigned int &, unsigned int ue2::NFAGraphVertexProps::*>, unsigned long, unsigned long &>, boost::iterator_property_map<std::__1::__wrap_iter<void **>, boost::adj_list_vertex_property_map<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, unsigned int, const unsigned int &, unsigned int ue2::NFAGraphVertexProps::*>, void *, void *&>, std::__1::vector<void *, std::__1::allocator<void *> >, boost::associative_property_map<boost::unordered::unordered_map<void *, void *, boost::hash<void *>, std::__1::equal_to<void *>, std::__1::allocator<std::__1::pair<void *const, void *> > > > >' requested here
    boost_ue2::lengauer_tarjan_dominator_tree(g, source, index_map, dfnum_map,
               ^
/tmp/hyperscan-20161015-47492-up8wy8/hyperscan-4.3.1/src/nfagraph/ng_dominators.cpp:81:12: note: in instantiation of function template specialization 'ue2::calcDominators<boost::reverse_graph<boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS>, const boost::adjacency_list<boost::listS, boost::listS, boost::bidirectionalS, ue2::NFAGraphVertexProps, ue2::NFAGraphEdgeProps, boost::no_property, boost::listS> &> >' requested here
    return calcDominators(boost::reverse_graph<NFAGraph, const NFAGraph &>(g.g),
           ^
/usr/local/include/boost/graph/detail/adjacency_list.hpp:1087:5: note: candidate template ignored: could not match 'undirected_graph_helper' against 'reverse_graph'
    degree(typename Config::vertex_descriptor u,
    ^
/usr/local/include/boost/graph/detail/adjacency_list.hpp:1514:5: note: candidate template ignored: could not match 'bidirectional_graph_helper_with_property' against 'reverse_graph'
    degree(typename Config::vertex_descriptor u,
    ^
2 errors generated.
make[2]: *** [CMakeFiles/hs.dir/src/nfagraph/ng_dominators.cpp.o] Error 1
make[1]: *** [CMakeFiles/hs.dir/all] Error 2
make: *** [all] Error 2
/usr/local/Homebrew/Library/Homebrew/debrew.rb:11:in `raise'

Full build log is here: https://gist.github.com/ilovezfs/3b8db4415d055a04cf0099e585ad12f3

(Resubmitting the PR against 'develop' branch as suggested in #286).

This PR brings the ARM NEON port up to par with the x86. It is based on PR272 (#272), however it can be pulled in on its own as it includes the changes. All tests pass on both ARM and x86 (obviously, we wouldn't want to break x86 functionality). As already described in PR272, x86-specific code has been abstracted away and arch specific code has been put into separate headers in src/util/arch/. This way, maintining arch-specific code of hyperscan and/or porting hyperscan to a new architecture (eg Power/RISC-V/etc) should now be much easier. Please consider merging this PR.

Hi,

I'm trying to evaluate the performance of hyperscan based on https://www.hyperscan.io/2017/05/08/performance-analysis-hyperscan-hsbench/

While the below download link for sample data is missing on 01.org https://01.org/downloads/sample-data-hyperscan-hsbench-performance-measurement

Where can I find them?

Thanks.

Hello,

I believe I have found a bug when running Hyperscan on Windows. On linux I have not been able to reproduce the error. It happens when I try to free a serialized database without a custom memory allocator. When I set a custom memory allocator (which is only a malloc and a free) it is able to free without further problem, however without defining this custom allocator it crashes. By using DrMemory I was able to find the following memory errors:

Error #2: INVALID HEAP ARGUMENT to free 0x0000000002069f30
# 0 replace_free               [D:\a\drmemory\drmemory\common\alloc_replace.c:2710]
# 1 main   
Note: @0:00:07.101 in thread 18456
Note: refers to -1 byte(s) before next malloc
Note: next higher malloc: 0x0000000002069f30-0x000000000206b3e0
Note: refers to -5296 byte(s) beyond last valid byte in prior malloc
Note: prev lower malloc:  0x0000000002069f30-0x000000000206b3e0

Error #3: LEAK 5296 direct bytes 0x0000000002069f30-0x000000000206b3e0 + 0 indirect bytes
# 0 replace_malloc                      [D:\a\drmemory\drmemory\common\alloc_replace.c:2580]
# 1 hs_runtime.dll!hs_serialize_database+0x93     (0x00007ffa94882b44 <hs_runtime.dll+0xc2b44>)
# 2 main   

(Note that Error #1 is unrelated)

It seems I am not able to free the serialized database returned by hs_serialize_database. Nevertheless, if I set the custom allocator to the following one:

void * allocator(size_t bytes)
{
    return (void *) malloc(bytes);
}

void deallocator(void * ptr)
{
    free(ptr);
}

[...]
hs_err = hs_set_misc_allocator(allocator, deallocator);
    if(hs_err != HS_SUCCESS)
        throw std::runtime_error("ERROR: Could not use allocator");
[...]

Then no leak or invalid heap argument is reported and the program finishes correctly.

Version of the build and reproducible example

I have built Hyperscan 5.4.0 in Cygwin64 version 3.3.4(0.341/5/3), Windows 11 Enterprise version 21H2 and GCC version x86_64-w64-mingw32-g++ 11.2.0. An isolated (hopefully reproducible) example is shown below:

#include <iostream>
#include <vector>
#include <hs.h>

void * allocator(size_t bytes)
{
    return (void *) malloc(bytes);
}

void deallocator(void * ptr)
{
    free(ptr);
}

int main(int ac, char ** av)
{
    // Vectors to hold the patterns expressions, flags and IDs
    std::vector<const char *> cstr_patterns;
    std::vector<unsigned> patterns_flags;
    std::vector<unsigned> patterns_ids;

    std::string p1 = "A random regex pattern";
    std::string p2 = "value: [0-9]+";

    cstr_patterns.push_back(p1.c_str());
    cstr_patterns.push_back(p2.c_str());
    patterns_flags.push_back(HS_FLAG_DOTALL | HS_FLAG_MULTILINE | HS_FLAG_SOM_LEFTMOST);
    patterns_flags.push_back(HS_FLAG_DOTALL | HS_FLAG_MULTILINE | HS_FLAG_SOM_LEFTMOST);
    patterns_ids.push_back(0);
    patterns_ids.push_back(1);

    // Database structures
    hs_database_t * db_block = NULL;
    hs_compile_error_t * compile_err = NULL;

    std::cout << "Begin compilation" << std::endl;
    int err = 0;
    
    // Compile
    err = hs_compile_multi(cstr_patterns.data(), patterns_flags.data(),
                    patterns_ids.data(), cstr_patterns.size(), HS_MODE_BLOCK,
                    NULL, &db_block, &compile_err);
    
    // Check if there were any errors
    if (err != HS_SUCCESS)
    {
        std::string error_msg;
        if (compile_err->expression < 0)
            error_msg.append("ERROR: "+std::string(compile_err->message)+"\n"); 
        else
            error_msg.append("ERROR: Pattern [id: "+std::string(std::to_string(compile_err->expression))+"] "+std::string(cstr_patterns[compile_err->expression])
                + " failed with error "+std::string(compile_err->message)+"\n");
        hs_free_compile_error(compile_err);
        throw std::runtime_error(error_msg);
    }
    else
        hs_free_compile_error(compile_err);

    std::cout << "Finished compilation" << std::endl;
    
    // Set custom allocator
    hs_error_t hs_err;
    hs_err = hs_set_misc_allocator(allocator, deallocator);
    if(hs_err != HS_SUCCESS)
        throw std::runtime_error("ERROR: Could not use allocator");
    

    std::cout << "Begin serialization " << std::endl;

    // Serialize DB
    size_t serialized_size;
    char * serialized_db = NULL;

    hs_err = hs_serialize_database(db_block, &serialized_db, &serialized_size);
    if(hs_err != HS_SUCCESS)
        throw std::runtime_error("ERROR: Could not serialize db");

    std::cout << "Finished serialization " << std::endl;
    
    // Free database
    std::cout << "Freeing database " << std::endl;
    hs_err = hs_free_database(db_block);
    if(hs_err != HS_SUCCESS)
        throw std::runtime_error("ERROR: Could not free db");
    
    // Do something .........

    // Free serialized db
    std::cout << "Freeing serialized database" << std::endl;
    free(serialized_db);
    
    std::cout << "Finished" << std::endl;

    return 0;
}

Also the full compilation line (on cygwin):

 /bin/x86_64-w64-mingw32-g++.exe -static -g -fno-inline -fno-omit-frame-pointer -o isolated.exe -I/<path-to-hyperscan>/hyperscan/src/ -I/<path-to-hyperscan>/hyperscan/ isolated.cpp /<path-to-hyperscan>/hyperscan/bin/hs.dll /<path-to-hyperscan>/hyperscan/bin/hs_runtime.dll

In order to reproduce the error, simply comment the lines regarding the setting of the hs_set_misc_allocator. I believe the problem is that the pointer returned by char *out = hs_misc_alloc(length); in database.c is not freeable by the default system free.

Thank you for your support, Esteban

Hi, everyone. I try to use my custom corpora and simple pattern to benchmark by hsbench, it will abort(RelWithDebug) or scan fail(Release) in literal mode.

My pattern:

# will error
1:/xt>DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD/i
# will error
# 1:/xt<DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD/i
2:/123/i

I have test in some case:

• In pattern id=1, either > or <, it will error.
• In pattern id=1, if i reduce the number of D, or replace the last D to A, it works ok.
• Without id=2, it works ok.
• In non-literal mode, it works ok.
• In pattern id=1, replace xt to xy, it works ok

The error code is: src/rose/program_runtime.c +3492 In function: roseRunProgram_l

This is my corpora: 23414.zip

I want to know why and how to fix it, really thanks for everyone first.

When compiling the project under macOS with the latest command line tools version 13.3.0.0.1.1645755326, which uses clang version Apple clang version 13.1.6 (clang-1316.0.21.2), the project fails to be built, reporting error like:

FAILED: CMakeFiles/hs_compile.dir/src/nfa/mcsheng_compile.cpp.o 
/Library/Developer/CommandLineTools/usr/bin/c++  -I/Users/ss/dev/tools/vcpkg/buildtrees/chimera/x64-osx-dbg -I/Users/ss/dev/tools/vcpkg/buildtrees/chimera/src/28a3d3326e-133ce01161.clean -I/Users/ss/dev/tools/vcpkg/buildtrees/chimera/src/28a3d3326e-133ce01161.clean/src -isystem /Users/ss/dev/tools/vcpkg/buildtrees/chimera/src/28a3d3326e-133ce01161.clean/include -isystem /Users/ss/dev/tools/vcpkg/installed/x64-osx/include -fPIC -march=native -mtune=native -O2 -std=c++11 -Wall -Wextra -Wshadow -Wswitch -Wreturn-type -Wcast-qual -Wno-deprecated -Wnon-virtual-dtor -fno-strict-aliasing -Werror -fno-omit-frame-pointer -Wvla -Wpointer-arith -Wno-self-assign -Wno-unused-const-variable -Wno-ignored-attributes -Wno-redundant-move -Wweak-vtables -Wmissing-declarations  -g -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk -MD -MT CMakeFiles/hs_compile.dir/src/nfa/mcsheng_compile.cpp.o -MF CMakeFiles/hs_compile.dir/src/nfa/mcsheng_compile.cpp.o.d -o CMakeFiles/hs_compile.dir/src/nfa/mcsheng_compile.cpp.o -c /Users/ss/dev/tools/vcpkg/buildtrees/chimera/src/28a3d3326e-133ce01161.clean/src/nfa/mcsheng_compile.cpp
/Users/ss/dev/tools/vcpkg/buildtrees/chimera/src/28a3d3326e-133ce01161.clean/src/nfa/mcsheng_compile.cpp:1170:13: error: variable 'total_daddy' set but not used [-Werror,-Wunused-but-set-variable]
        u16 total_daddy = 0;
            ^
1 error generated.

And it is caused by clang-13 introducing -Wunused-but-set-variable by default, which makes compilation fails. Similar with this issue

hi, everybody I am using hyperscan do some serious heavy-load query works in c++. For the best performance, our project hire session-aware-memory-allocation technology, just like Google's Arena. pls see Protobuf's Arena Document. I noticed that hyperscan library support use customized allocation function. from hs_set_allocator but this API was not designed for our scenairo, we cannot passing a c++ object's member funtion as c-style-function-ptr, so I need an another parameter just like a void*, then I can pass the Arena object ptr into the stateless global allocation function. hs_error_t hs_set_allocator(void*(size_t, void*) alloc_func, void* ctx, void(void*, void*) free_func, void* ctx) I hope u guys like this idea, and implement it, it can really boost hyperscan in some heavy-load quering services.

Hello there,

I was experimenting with scratch buffer allocation and noticed that one specific pattern would cause a crash. The code I used is as follow, I have 2 similar patterns and only allocate the scratch buffer for pattern1 but not pattern2:

#include <hs/hs_compile.h>
#include <stdio.h>
#include <string.h>
#include <hs/hs.h>

static int eventHandler(unsigned int id, unsigned long long from,
                        unsigned long long to, unsigned int flags, void *ctx) {
    printf("Match for pattern \"%s\" at offset %llu\n", (char *)ctx, to);
    return 0;
}

int main(int argc, char *argv[]) {
    const char *pattern1 = "^(ftps?|https?).*?\\?+$";
    const char *pattern2 = "^(ftps?|https?)://.*$";

    hs_compile_error_t *compile_err;
    hs_database_t *database1;
    hs_compile(pattern1, HS_FLAG_SOM_LEFTMOST, HS_MODE_BLOCK, NULL, &database1,
                   &compile_err);

    hs_database_t *database2;
    hs_compile(pattern2, HS_FLAG_SOM_LEFTMOST, HS_MODE_BLOCK, NULL, &database2,
                   &compile_err);

    const char *inputData = "https://foo.bar/";
    hs_scratch_t *scratch = NULL;
    hs_alloc_scratch(database1, &scratch);
    hs_scan(database1, inputData, strlen(inputData), 0, scratch, eventHandler,
                (void*)pattern1);
    hs_scan(database2, inputData, strlen(inputData), 0, scratch, eventHandler,
                (void*)pattern2);

    return 0;
}

Expected results: the second hs_scan should return an error, since the scratch buffer was not allocated properly for that pattern Actual results: the second hs_scan is SEGV, here is the stack trace:

#0  0x00007ffff7c3b5cd in memset (__len=32, __ch=0, __dest=0x0)
    at /usr/include/x86_64-linux-gnu/bits/string_fortified.h:71
#1  fatbit_clear (bits=0x0) at ./src/util/fatbit.h:60
#2  recordAnchoredLiteralMatch (end=8, anch_id=0, scratch=0x7fffcc00f240, t=<optimized out>)
    at ./src/rose/program_runtime.c:101
#3  roseRunProgram (t=<optimized out>, scratch=<optimized out>, programOffset=<optimized out>, som=0,
    end=<optimized out>, prog_flags=<optimized out>) at ./src/rose/program_runtime.c:2278
#4  0x00007ffff7c22fd0 in roseAnchoredCallback ([email protected]=0, end=<optimized out>,
    [email protected]=1888, [email protected]=0x7fffcc00f240) at ./src/rose/match.c:215
#5  0x00007ffff7bd5a04 in doComplexReport (cached_accept_id=<synthetic pointer>,
    cached_accept_state=<synthetic pointer>, eod=0 '\000', loc=8, s=21, m=<optimized out>,
    ctxt=0x7fffcc00f240, cb=0x7ffff7c22f70 <roseAnchoredCallback>) at ./src/nfa/mcclellan.c:74
#6  mcclellanExec8_i (mode=CALLBACK_OUTPUT, c_final=0x0, single=0 '\000', ctxt=0x7fffcc00f240,
    cb=0x7ffff7c22f70 <roseAnchoredCallback>, offAdj=0, len=8, buf=0x7fffd4014580 "https://foo.bar/",
    state=<synthetic pointer>, m=<optimized out>) at ./src/nfa/mcclellan.c:502
#7  nfaExecMcClellan8_Bi (single=0 '\000', context=0x7fffcc00f240,
    length=8, buffer=0x7fffd4014580 "https://foo.bar/",
    offset=0, n=<optimized out>) at ./src/nfa/mcclellan.c:922
#8  nfaExecMcClellan8_B (n=<optimized out>, offset=0, buffer=0x7fffd4014580 "https://foo.bar/", length=8,
    cb=0x7ffff7c22f70 <roseAnchoredCallback>, context=0x7fffcc00f240) at ./src/nfa/mcclellan.c:945
#9  0x00007ffff7c0dbe1 in runAnchoredTableBlock (scratch=0x7fffcc00f240, atable=<optimized out>,
    t=0x555557cfde00) at ./src/rose/block.c:63
#10 roseBlockAnchored (scratch=0x7fffcc00f240, t=0x555557cfde00) at ./src/rose/block.c:212
#11 roseBlockExec (t=<optimized out>, scratch=<optimized out>) at ./src/rose/block.c:395
#12 0x00007ffff7b44f4f in rawBlockExec (scratch=0x7fffcc00f240, rose=0x555557cfde00)
    at ./src/runtime.c:188
#13 hs_scan (db=<optimized out>, data=<optimized out>, length=15, flags=<optimized out>,
    scratch=0x7fffcc00f240, onEvent=<optimized out>, userCtx=<optimized out>) at ./src/runtime.c:419

Note: there is no crash if I remove the HS_FLAG_SOM_LEFTMOST flag. Workaround: reallocating the scratch buffer with the second pattern solves the issue.

It sounds to me that some sanity checks are missing inside hs_scan.