This PR addresses a few things: Meson support, a fix to parse_RFC_850(), re-add support for code coverage in CMake and update Travis config. All the details about this four things are in their respective commit description :)

If you don't know what Meson is, I highly recommend you to check it out. Meson build files are far more readable and maintainable than CMake ones, and the build system is also faster, which is never a bad thing.

I have actually finished my program and it does exactly what I want, but I have been doing it in codeblocks on a Ubuntu system. So now I want to run my program on a Centos headless server. I have made a Makefile and my Makefile works on Ubuntu without errors, but then pistache is installed.

So I was wondering if someone could talk me through the process of installing Pistache on Centos. Is there a yum command to install pistache or do I just clone pistache from git and use the Makefile to build it and then copy the pistache folder to my project?

Bear in mind I am new to linux and I usually do everything in windows, so if my question sounds stupid then that is why.

1. Keep-alive connection server endpoint, send 408 message after idle timeout, it's not expected.
2. I add an keep-alive timeout option. Before starting the HTTP service, you can set the keepalive time for the long connection. If the keepalive time exceeds this value, you can disable the long connection.
3. I use tcpdump capture packet, confirming that 408 is not sent and the connection is directly closed.

This new CI job builds and tests Pistache on different Linux distros, with different compilers, and under different sanitizers, and also re-adds Codecov coverage integration back.

Currently, Pistache is tested on Debian Stable, Debian Testing and Red Hat Enterprise Linux 8, and the code is built with GCC and Clang. Sanitizers are only used on Debian, because as far as I'm aware RHEL doesn't ship the various sanitizers in its base images.

ThreadSanitizer is also disabled for now, as it reports a lot of data races in some tests. Some of the bugs are in the unit tests themselves, but I haven't yet looked into solving them. @dennisjenkins75 I seem to remember that you opened a meta-issue about dace conditions in Pistache but I haven't found it.

~I'll add RHEL 9 as soon as Red Hat publishes the image on Docker Hub~ added using Red Hat's own registry

It would be nice to add something like Gentoo too; @dennisjenkins75 you use/used Gentoo, right? Would you be able to help?

~I've also fixed a bug in the streaming_test test that caused lock ups on old distros (@kiplingw this means that PPA builds for Ubuntu 20.10 and older should be fixed as well)~ pushed as 8ce07256dc220cbf96b3fd6f5991a3c68fd54aba

Lastly, I ~removed~ reduced the scope of the autopkgtest job; you can find my rationale in the commit message.

I'm also currently facing the issue described in #1007. I added the MSG_NOSIGNAL Flag as proposed by @amang8662. But there was still the problem, how to get the info if the socket is still alive. So I checked on the documentation (https://linux.die.net/man/3/getsockopt). With the function getsockopt() you are able to receive the last error of the socket. In case the pipe breaks, the error is returned through this function. So we can check if the stream is still alive by testing the return value.

I added the function isOpen() which returns true if there isn't an error on the socket and false otherwise.

This is my first PR in any open-source project. So I hope for leniency.

• Changed ingestion buffer to std::vector instead of a stack-based array. There will be a performance penalty, but it allows for file uploads
• Added Const::MaxPayload to avoid killing hosts due to massive payloads.

Introduction

This patch introduces full SSL support for the pistache endpoint. The purpose of this patch is to be able to use SSL encryption without a SSL proxy, which is pretty useful for embedded development.

It has been tested locally with simple SSL support and certificate connection, my example file is below. I did try to match the coding style of pistache in order to ease the merge, apologies if I missed anything.

Features

First of all, in order to enable SSL support in the library, one has to compile with PISTACHE_USE_SSL option ON. One can then configure the endpoint to accept / use SSL connections only:

Http::Endpoint server(addr);
server.useSSL("./server.crt", "./cert/server.key");

With that done, all the connections to the server shall now be in HTTPS.

One can also enable certificate authentication against the server:

server.useSSLAuth("./rootCA.crt");

The server will now only accept client connection with a certificate signed with the Certificate Authority passed in the example above.

Tests

Full test file:

#include <pistache/endpoint.h>

using namespace Pistache;

struct HelloHandler : public Http::Handler {
    HTTP_PROTOTYPE(HelloHandler)

    void onRequest(const Http::Request& request, Http::ResponseWriter writer) {
        writer.send(Http::Code::Ok, "Hello, World!");
    }
};

int main(void) {
    Pistache::Address   addr;
    auto opts = Http::Endpoint::options().threads(1);

    addr = Pistache::Address(Pistache::Ipv4::any(), Pistache::Port(9080));

    Http::Endpoint server(addr);
    server.init(opts);
    server.setHandler(std::make_shared<HelloHandler>());
    server.useSSL("./cert/server.crt", "./cert/server.key");

    server.serve();
    return 0;
}

Compiled with:

g++ main.cpp -L../pistache/build/src/ -lpistache -I../pistache/include/ -o server -lpthread -lssl -lcrypto

In order to generate a Certificate Authority and server / client certificate and key, please refer to this gist

SSL

Simple CURL:

$> curl https://127.0.0.1:9080/test --cacert CA/rootCA.crt -k                        
Hello, World!% 

With CA/rootCA.crt as my root Certificate Authority, used to generate the server certificate. The -k is to ignore the curl error:

curl: (51) SSL: certificate subject name 'Common Name Test Server' does not match target host name '127.0.0.1'

Since I did not generate a server certificate for 127.0.0.1

SSL authentication

With the following line added in my test file:

server.useSSLAuth("./CA/rootCA.crt");

Simple CURL:

$> curl https://127.0.0.1:9080/test --cacert CA/rootCA.crt -k
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

With proper client cert & key:

$> curl https://127.0.0.1:9080/test --cacert CA/rootCA.crt --cert ./client_cert/client.crt --key client_cert/client.key -k
Hello, World!%

Same usage of the -k option as above.

Verbose output

$> curl https://127.0.0.1:9080/test --cacert CA/rootCA.crt --cert ./client_cert/client.crt --key client_cert/client.key -k -v
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 9080 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: CA/rootCA.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=FR; ST=Paris; L=Paris; O=XXX; CN=Common Name Test Server
*  start date: May 29 15:59:03 2018 GMT
*  expire date: Oct 11 15:59:03 2019 GMT
*  issuer: C=FR; ST=Paris; L=Paris; O=XXX; CN=Common Name Root Authority; [email protected]
*  SSL certificate verify ok.
> GET /test HTTP/1.1
> Host: 127.0.0.1:9080
> User-Agent: curl/7.60.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Length: 13
< 
* Connection #0 to host 127.0.0.1 left intact
Hello, World!% 

I am trying to write a c++ pistache server that, on a specific endpoint, has to contact another pistache server. This is the scenario:

client -> server1 -> server2

client <- server1 <- server2

I am having problems waiting for the response in server1 and sending it back to the client asynchronously.

I share my server1 code to show you how I did implement it.

void doSmth(const Rest::Request& request, Http::ResponseWriter httpResponse){
        auto resp_srv2 = client.post(addr).body(json).send();    
        resp_srv2.then(
        [&](Http::Response response) {
            do_smth;
        },
        [&](std::exception_ptr exc) {
            PrintException excPrinter;
            excPrinter(exc);
    });

    Async::Barrier<Http::Response> barrier(resp_srv2);
    barrier.wait_for(std::chrono::seconds(1));        
    httpResponse.send(response_code);
}

In particular this code works only if I set one single thread for the server1. If I use more thread it crash on the first request. Of course the performances are not that good.

If I try to follow your example, I am able to use more threads. But this has a problem (overflow?) with requests count. On my sever, I can reach around 28k requests sent from the client to server1, and after that it crashes.

Also, if I do like:

               resp_srv2.then(
		    [&](Http::Response response) {
		        httpResponse.send(r.response_code);             
		    },
		    [&](std::exception_ptr exc) {
		        PrintException excPrinter;
		        excPrinter(exc);
		});

it does not work well giving back a segmentation fault when the method is called.

Am I doing something wrong? Please, help me to fix this issue.

Fixes #858.

As per the recommendation by @kiplingw in https://github.com/pistacheio/pistache/issues/858#issuecomment-758279752 this PR upgrades from C++14 to C++17 to take advantage of std::optional.

This PR complements #859, and should be merged only after merging that one.

I recommend reviewing this adding ?w=1 at the end of the URL to ignore whitespace changes.

Fixes #826, fixes #835.

When used like &vec[0], there would be an assert in gnu std lib:

/usr/include/c++/8/bits/stl_vector.h:932: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) [with _Tp = char; _Alloc = std::allocator; std::vector<_Tp, _Alloc>::reference = char&; std::vector<_Tp, _Alloc>::size_type = long unsigned int]: Assertion '__builtin_expect(__n < this->size(), true)' failed.

on the other hand, using .data() is allowed even on empty uninited vectors

array<> access is modified for consistency.

Hey there!

Thanks a lot for the documentation. I am currently testing some features and i found a missing bracket. Under "Static file serving":

if (request.resource() == "/doc" && request.method == Http::Method::Get) {
    Http::serveFile(response, "README.md");
}

"request.method" has to be "request.method()"

Maybe it will help others.

Issue #1103 : please check pistachio/pistache issue list I just create a UT that include a blocking issue when removing route, just to let people get my PR onto their development box and dig into the problem.

This may be related to #708 or #761, but I've noticed an issue recently with one of my servers that appears to hang at the acceptThread.join() call here:

    Listener::~Listener()
    {
        if (isBound())
            shutdown();
        if (acceptThread.joinable())
            acceptThread.join();

        if (listen_fd >= 0)
        {
            close(listen_fd);
            listen_fd = -1;
        }
    }

This appears to happen occasionally on destruction via Pistache::Http::Endpoint::~Endpoint(). The acceptThread.join() is just a wrapper for std::thread::join(), of course.

I have two separate core dumps of the same process. One before I asked it to terminate gracefully via SIGTERM signal and the other after issuing the signal. In both I see a Pistache listener thread that appears to be hanging in OpenSSL's BIO_read(). I'm pretty sure this is the problem because in the coredump post SIGTERM after all other threads, besides main, have exited, still remains:

#0  __GI___libc_read (nbytes=5, buf=0x7ffb94008da3, fd=4) at ../sysdeps/unix/sysv/linux/read.c:26
        sc_ret = -512
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  __GI___libc_read (fd=4, buf=0x7ffb94008da3, nbytes=5) at ../sysdeps/unix/sysv/linux/read.c:24
#2  0x00007ffb9d5f0ddd in  () at /lib/x86_64-linux-gnu/libcrypto.so.3
#3  0x00007ffb9d5e12eb in  () at /lib/x86_64-linux-gnu/libcrypto.so.3
#4  0x00007ffb9d5e42e5 in  () at /lib/x86_64-linux-gnu/libcrypto.so.3
#5  0x00007ffb9d5e4467 in BIO_read () at /lib/x86_64-linux-gnu/libcrypto.so.3
#6  0x00007ffb9cba934c in  () at /lib/x86_64-linux-gnu/libssl.so.3
#7  0x00007ffb9cbaaf19 in  () at /lib/x86_64-linux-gnu/libssl.so.3
#8  0x00007ffb9cbbcb36 in  () at /lib/x86_64-linux-gnu/libssl.so.3
#9  0x00007ffb9cbbe608 in  () at /lib/x86_64-linux-gnu/libssl.so.3
#10 0x00007ffb9d4aeb2d in Pistache::Tcp::Listener::handleNewConnection() (this=0x7fff974787f8) at ../src/server/listener.cc:481
        ssl_data = 0x7ffb94011320

                  peer_addr = {ss_family = 2, __ss_padding = "\302v<\331KF\000\000\000\000\000\000\000\000@FǙ\373\177\000\000\364M\362\234\373\177", '\000' <repeats 18 times>, "\240B\033\235\373\177\000\000\200\020\000\224\373\177\000\000\200#\000\000\000\000\000\000HiǛ\373\177\000\000\000\000\000\000\000\000\000\001\000\020\000\000\000\000\000\000\000@G\231\373\177", '\000' <repeats 11 times>, "\020\200\000\000\000\000", __ss_align = 4096}
        client_fd = 4
        ssl = 0x0
        peer = std::shared_ptr<Pistache::Tcp::Peer> (use count -1811939008, weak count 32762) = {get() = 0x7ffb94023bf0}
        peer_alias = <optimized out>
#11 0x00007ffb9d4aeef3 in Pistache::Tcp::Listener::run() (this=0x7fff974787f8) at ../src/server/listener.cc:367
        fd = <optimized out>
        event = @0x7ffb94001230: {flags = {val = Pistache::Polling::NotifyOn::Read}, tag = {value_ = 5}}
        __for_range = std::vector of length 1, capacity 1 = {{flags = {val = Pistache::Polling::NotifyOn::Read}, tag = {value_ = 5}}}
        events = std::vector of length 1, capacity 1 = {{flags = {val = Pistache::Polling::NotifyOn::Read}, tag = {value_ = 5}}}
        ready_fds = <optimized out>
#12 0x00007ffb9d1b42b3 in  () at /lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x00007ffb9cf24b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
        ret = <optimized out>
        pd = <optimized out>

                      unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140735731431920, 1193485945046415927, 140718627063360, 0, 140718646642768, 14073573143227--Type <RET> for more, q to quit, c to continue without paging--c
2, -1195940266122823113, -1195946767574910409}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#14 0x00007ffb9cfb6a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

The acceptThread.join() is probably deadlocked waiting for that listener thread to finish what it's doing. The ../src/server/listener.cc:481 is the call to SSL_accept(ssl_data). So the handshake appears to hang every now and then, rather than timing out.

This then raises the question of what is making SSL_accept(ssl_data) hang indefinitely? This may have always been a problem, but perhaps something in my network topology my server sits in recently made this problem become evident. Perhaps the workaround is to specify a timeout?

This person's issue seems very similar to mine in that as soon as the hang occurs the server will not accept any new connections.

Hi, I'm unable to get any request coming from one specific client and upon using wireshark, I was able to find why on_request never got hit. Pistache Automatically sent the following response HTTP/1.1 415 Unsupported Media Type\r\n Further Explanation attached to it was : "Malformed Media Type, expected a '/' after the top type"

The Accept Tag of Request is as follows: Accept: text/html, image/gif, image/jpeg, *; q=.2, * / *; q=.2\r\n (Edit: I myself gave space in * / * , as it italicized the query)

Upon analyzing, void Accept::parseRaw(const char* str, size_t len) in http_header.cc is handling eof and ',' in while conditions and not semicolon ';' whereas the Accept has semicolons as well along with q=.2

Furthermore it seems that it is going in mime.cc and on this check if (!match_literal('/', cursor)) , it raised the error , and just a thought, can it be caused because of the single '*' after jpeg?

Kindly, can you guide me what can be the problem and how can it be solved? I might not be able to change the Accept tag as it's Pre/By Default defined in a Cisco UCCX Telephony setup

I started writing a thread-safe wrapper for Router / Private::RouterHandler to support adding and removing routes dynamically, when I saw in the RouterHandler comments:

            /**
             * Used for mutable router. Useful if it is required to
             * add/remove routes at runtime.
             * \param[in] router Pointer to a (mutable) router.
             */
            explicit RouterHandler(std::shared_ptr<Rest::Router> router);

I wondered how this might be done safely, once the server is running? Can it only be done when processing a request on a single-threaded server?

I'm trying to load a .csv file (Content-Disposition: attachment) when accessing the endpoint, but the tools I found for this (response.send() and serveFile() do not provide this capability.) How can I do it?