I'm trying to implement a connection pool for the http client. I'm attempting to store for every remote address (host:port) a queue of HTTPUpstreamSessions that I can reuse. I'm having issues reusing those sessions. Looking through all handlers and callbacks of HTTPTransaction and HTTPUpstreamSession, I can't find a callback in which I can put the HTTPUpstreamSession back into the pool and have it reused, for a new transaction. Furthermore, if one of the transactions in the session receives a timeout, the session is destroyed.

The performance for creating a new Session and transaction for every outbound request is pretty bad, which defeats the purposed of switching to Proxygen. Can anyone please point me to the right direction? Or there needs to be some code change for that?

PS: Using regular HTTP 1.1, no ssl, simple request/response.

Thanks

I have written a HTTP router by extending RequestHandlerFactory to multiplex paths to RequestHandler implementations. The routing uses a compacted prefix tree.

Is this something you are interested in having in your repo or should I create a separate repo for it? If you are interested then I will merge it with tests into a public fork and then submit a PR.

Question

We would like to implement a Http-Client which can cache the session and reuse the same HTTPSession for sub-sequent http-requests so, we can avoid new connection-creation for every new request. I have followed the example provided at CurlClientMain.cpp.

Issue: I am not able to use created session (proxygen::HTTPUpstreamSession) more than once. HTTPConnector creates a session for very first time and creates a txn on it to serve a http-request. Once, given http-request executes fully, I am trying to reuse the same session by checking if that session is reusable using below condition. proxygen::HTTPUpstreamSession::isReusable() But once, a first txn completes it shutdowns the session and it's not reusable anymore and function call on session.isReusable() gives segmentation fault as session is already shutdown.

Stacktrace:

• I have tried to debug the issue and realized as soon as txn completes it shutdowns the session and I can get below stacktrace.
• So, is there any way, I can create a persistent-session and cache the session and reuse it to create new transaction again to serve new http-request?

Tag: v2018.04.02.00

Thread 1 "serverTest" hit Breakpoint 1, proxygen::HTTPSession::checkForShutdown ([email protected]=0x7e1ad0) at session/HTTPSession.cpp:2092
2092	in session/HTTPSession.cpp
(gdb) bt
#0  proxygen::HTTPSession::checkForShutdown ([email protected]=0x7e1ad0) at session/HTTPSession.cpp:2092
#1  0x00007ffff785863b in proxygen::HTTPSession::shutdownTransport ([email protected]=0x7e1ad0, shutdownReads=<optimized out>, [email protected]=false, 
    shutdownWrites=<optimized out>, [email protected]=true, errorMsg="") at session/HTTPSession.cpp:2027
#2  0x00007ffff7861027 in proxygen::HTTPSession::detach (this=0x7e1ad0, txn=<optimized out>) at session/HTTPSession.cpp:1607
#3  0x00007ffff7830f02 in proxygen::HTTPTransaction::onDelayedDestroy (this=0x7e0928, delayed=<optimized out>) at session/HTTPTransaction.cpp:110
#4  0x00007ffff78365a5 in proxygen::HTTPTransaction::processIngressEOM (this=0x7e0928) at session/HTTPTransaction.cpp:442
#5  0x00007ffff7837ae5 in proxygen::HTTPTransaction::onIngressEOM ([email protected]=0x7e0928) at session/HTTPTransaction.cpp:437
#6  0x00007ffff785d83d in proxygen::HTTPSession::onMessageComplete (this=0x7e1ad0, streamID=<optimized out>, upgrade=<optimized out>)
    at session/HTTPSession.cpp:853
#7  0x00007ffff7883690 in proxygen::HTTP1xCodec::onMessageComplete ([email protected]=0x7e0820) at codec/HTTP1xCodec.cpp:1079
#8  0x00007ffff78838ea in proxygen::HTTP1xCodec::onMessageCompleteCB (parser=<optimized out>) at codec/HTTP1xCodec.cpp:1232
#9  0x00007ffff7845629 in proxygen::http_parser_execute ([email protected]=0x7e0838, settings=<optimized out>, 
    [email protected]=0x7e3110 "HTTP/1.1 200 OK \r\nHost: 127.0.0.1:9999\r\nContent-Length: 63\r\nAccept: */*\r\nUser-Agent: my-agent-http\r\nConnection: keep-alive\r\n\r\n{ \"key\" : \"x\", \"sKey\" : \"y\", \"token\" : \"z\" }\r\n", [email protected]=189)
    at ../../external/http_parser/http_parser_cpp.cpp:1982
#10 0x00007ffff78856f8 in proxygen::HTTP1xCodec::onIngress (this=0x7e0820, buf=...) at codec/HTTP1xCodec.cpp:201
#11 0x00007ffff78568f9 in proxygen::HTTPSession::processReadData ([email protected]=0x7e1ad0) at session/HTTPSession.cpp:487
#12 0x00007ffff7856d13 in proxygen::HTTPSession::readDataAvailable (this=0x7e1ad0, readSize=189) at session/HTTPSession.cpp:440
#13 0x00007ffff6e35f22 in folly::AsyncSocket::handleRead (this=0x7e0500) at io/async/AsyncSocket.cpp:1900
#14 0x00007ffff6e30ff9 in folly::AsyncSocket::ioReady (this=0x7e0500, events=2) at io/async/AsyncSocket.cpp:1710
#15 0x00007ffff586aa14 in event_base_loop () from /lib64/libevent-2.0.so.5
#16 0x00007ffff6e57d1a in folly::EventBase::loopBody (this=0x7fffffffdce0, flags=<optimized out>) at io/async/EventBase.cpp:313

compile environment information: ubuntu16.04 64bit gcc-5.4 when i run proxygen deps.sh to compile proxygen. i got an error: libtool: link: ranlib .libs/libqcram.a libtool: link: rm -fr .libs/libqcram.lax libtool: link: ( cd ".libs" && rm -f "libqcram.la" && ln -s "../libqcram.la" "libqcram.la" ) /bin/bash ../../libtool --tag=CXX --mode=link g++ -std=gnu++14 -pthread -DLIBMC_FBTRACE_DISABLE -Wno-missing-field-initializers -Wno-deprecated -O3 -g -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2 -o CompressionSimulator codec/compress/test/HTTPArchive.o codec/compress/experimental/simulator/Main.o codec/compress/experimental/simulator/CompressionSimulator.o libqpack.la libqcram.la libproxygenhttp.la ../services/libproxygenservices.la -liberty -lwangle -lfolly -lz -lssl -lcrypto -levent -lgflags -lglog -L/usr/lib64 -lpthread -pthread -lfolly -lglog -ldouble-conversion -lboost_system -lboost_thread libtool: link: g++ -std=gnu++14 -pthread -DLIBMC_FBTRACE_DISABLE -Wno-missing-field-initializers -Wno-deprecated -O3 -g -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2 -o CompressionSimulator codec/compress/test/HTTPArchive.o codec/compress/experimental/simulator/Main.o codec/compress/experimental/simulator/CompressionSimulator.o -pthread ./.libs/libqpack.a -L/usr/lib64 ./.libs/libqcram.a ./.libs/libproxygenhttp.a ../services/.libs/libproxygenservices.a -liberty -lwangle -lz -lssl -lcrypto -levent -lgflags -lpthread -lfolly -lglog -ldouble-conversion -lboost_system -lboost_thread -pthread /usr/bin/ld: //usr/local/lib/libfolly.a(CacheLocality.cpp.o): undefined reference to symbol '[email protected]@GLIBC_2.2.5' //lib/x86_64-linux-gnu/libdl.so.2: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status Makefile:916: recipe for target 'CompressionSimulator' failed make[5]: *** [CompressionSimulator] Error 1 make[5]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib/http' Makefile:1144: recipe for target 'all-recursive' failed make[4]: *** [all-recursive] Error 1 make[4]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib/http' Makefile:661: recipe for target 'all' failed make[3]: *** [all] Error 2 make[3]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib/http' Makefile:512: recipe for target 'all-recursive' failed make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib' Makefile:418: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen' Makefile:350: recipe for target 'all' failed make: *** [all] Error 2

i decide to add -ldl LDFLAGS in somewhere CMakeLists.txt to deal with the issues. But i couldn't known how to add it . So everyone can help me,give some ideas,thanks very much.

Hello,

I'm getting an error w/ the HTTPSession.

Problem Setup:

1. Start a simple http server w/ say an artificially high number of threads 30 or whatever.
2. Don't respond to http callbacks for a while.. i.e:

//wait for a while
proxygen::ResponseBuilder(downstream_).status(200, "OK).sendEOM();

Expected output:

None, I just didn't expect the EventBase to crash.

Source of the problem

void
HTTPSession::scheduleWrite() {
  // Do all the network writes for this connection in one batch at
  // the end of the current event loop iteration.  Writing in a
  // batch helps us packetize the network traffic more efficiently,
  // as well as saving a few system calls.
  if (!isLoopCallbackScheduled() &&
      (writeBuf_.front() || !txnEgressQueue_.empty())) {
    VLOG(5) << *this << " scheduling write callback";
    sock_->getEventBase()->runInLoop(this);
  }
}

eventBase->runInLoop(this)

does this:

void EventBase::runInLoop(LoopCallback* callback, bool thisIteration) {
  DCHECK(isInEventBaseThread());
  callback->cancelLoopCallback();
  callback->context_ = RequestContext::saveContext();
  if (runOnceCallbacks_ != nullptr && thisIteration) {
    runOnceCallbacks_->push_back(*callback);
  } else {
    loopCallbacks_.push_back(*callback);
  }
}

So the issue would be simply fix by

auto eb = sock_->getEventBase();
eb->runInEventBase([this, ](){ 
    eb->runInLoop(this);
});

Any thoughts? Happy to make the PR.

Instead of removing Travis whatsoever, we could use the Travis to just build the Docker container, since it proves that the code is compiling fine.

However there's one problem: the build takes too long on free Travis, so I have a hard time validating this pull request. It timed out.

https://travis-ci.org/wkoszek/proxygen/builds/140990904

Does Facebook have a commercial Travis account? Would you be willing to accept the pull request and start building proxygen again?

I ran the commands as mentioned (run ./deps.sh and then ./reinstall.sh) but the installation fails. Here are the last few lines of the make output: https://gist.github.com/vaibhav-kaushal/586b51257494d56e673a

Basically it never gets out of that tests folder. Here is a screenshot:

I have waited for more than an hour, nothing happened. I tried reinstalling the things again and again but fails every time. I have an Ubuntu updated 14.04 (desktop). What might be the problem here?

Most of my SSL request work okay but I am getting this segfault which seem to be coming from libcrypto searches for similar error say it might be openssl thing and should change the lib to some other ?

https://gist.github.com/abhiguru/26eb6ac11f9b32f386578e97d940b55e

#0 0x00007ffff709d384 in RSA_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 No symbol table info available. #1 0x00007ffff7429356 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0 No symbol table info available. #2 0x00007ffff7427e92 in SSL_free () from /lib/x86_64-linux-gnu/libssl.so.1.0.0 No symbol table info available. #3 0x00007ffff79efa3f in folly::AsyncSSLSocket::closeNow (this=0x7fffe00f1540) at io/async/AsyncSSLSocket.cpp:307 dg = {dd_ = 0x7fffe00f19d8} #4 0x0000555555931563 in proxygen::HTTPSession::shutdownTransport ([email protected]=0x7fffe0008910, [email protected]=false, shutdownWrites=, [email protected]=true, errorMsg="") at session/HTTPSession.cpp:1956 guard = {dd_ = 0x7fffe00089a0} notifyEgressShutdown = false notifyIngressShutdown = false error = #5 0x000055555593bbaa in proxygen::HTTPSession::detach (this=0x7fffe0008910, txn=) at session/HTTPSession.cpp:1554 guard = {dd_ = 0x7fffe00089a0} it = {first = 1, second = warning: RTTI symbol not found for class 'folly::HHWheelTimer::Callback'

{folly::HHWheelTimer::Callback = {<boost::intrusive::list_base_hook<boost::intrusive::link_mode<(boost::intrusive::link_mode_type)2>, void, void>> = {<boost::intrusive::generic_hook<boost::intrusive::ci rcular_list_algorithms<boost::intrusive::list_node_traits<void*> >, boost::intrusive::dft_tag, (boost::intrusive::link_mode_type)2, (boost::intrusive::base_hook_type)1>> = {<boost::intrusive::node_holder< boost::intrusive::list_node<void*>, boost::intrusive::dft_tag, 1u>> = {<boost::intrusive::list_node<void*>> = {next_ = , prev_ = }, }, <boost::intrusive::hook_tags_definer<boost::intrusive::generic_hook<boost::intrusive::circular_list_algorithms<boost::intrusive::list_node_traits<void*>

, boost::intrusive::dft_tag, (boost::intrusive::link_mode_type)2, (boost::intrusive::base_hook_type)1>, 1u>> = {}, }, }, vptr.Callback = <vtable for folly::HHWheelTimer::Callback+16>, wheel = , expiration_ = {r = 0}, bucket = -1, context = std::shared_ptr (empty) 0x0}, folly::DelayedDestructionBase = {boost::noncopyable_::noncopyable = {}, vptr.DelayedDestructionBase = <vtable for proxygen::HTTPTransaction+232>, guardCount = 0}, rateLimitCallback_ = warning: RTTI symbol not found for class 'folly::HHWheelTimer::Callback'

{folly::HHWheelTimer::Callback = {<boost::intrusive::list_base_hook<boost::intrusive::link_mode<(boost::intrusive::link_mode_type)2>, void, void>> = {<boost::intrusive::generic_hook<boost::intrusive::ci rcular_list_algorithms<boost::intrusive::list_node_traits<void*> >, boost::intrusive::dft_tag, (boost::intrusive::link_mode_type)2, (boost::intrusive::base_hook_type)1>> = {<boost::intrusive::node_holder< boost::intrusive::list_node<void*>, boost::intrusive::dft_tag, 1u>> = {<boost::intrusive::list_node<void*>> = {next_ = , prev_ = }, }, <boost::intrusive::hook_tags_definer<boost::intrusive::generic_hook<boost::intrusive::circular_list_algorithms<boost::intrusive::list_node_traits<void *> >, boost::intrusive::dft_tag, (boost::intrusive::link_mode_type)2, (boost::intrusive::base_hook_type)1>, 1u>> = {}, }, }, vptr.Callback = <vtable for folly::HHWheelTimer::Callback+16>, wheel = , expiration_ = {r = 0}, bucket = -1, context = std::shared_ptr (empty) 0x0}, txn_ = warning: RTTI symbol not f ound for class 'folly::HHWheelTimer::Callback' }, deferredIngress_ = std::unique_ptr<std::queue<proxygen::HTTPEvent, std::deque<proxygen::HTTPEvent, std::allocatorproxygen::HTTPEvent > >> containing 0x0, maxDeferredIngress_ = 0, deferredEgressBody_ = {static kChainLengthNotCached = 18446744073709551615, options_ = {cacheChainLength = true}, chainLength_ = 0, head_ = std::unique_ptrfolly::IOBuf containing 0x0}, direction_ = proxygen::TransportDirection::UPSTREAM, id_ = 1, seqNo_ = 0, handler_ = , transport_ = , egressState_ = proxygen::HTTPTransactionEgressSMData::State::SendingDone, ingressState_ = proxygen::HTTPTransactionIngressSMData::State::ReceivingDone, timeout_ = {wheelTimerPtr_ = , wheelTimer_ = std::weak_ptr (empty) 0x0, wheelTimerGuard_ = {dd_ = }, defaultTimeoutMS_ = {r = 140737140213136}}, stats = , recvWindow = {outstanding_ = 0, capacity_ = 65536}, sendWindow_ = {outstanding_ = 0, capacity_ = 1048576}, transportCallback_ = , trailers_ = std::unique_ptrproxygen::HTTPHeaders containing 0x0, chunkHeaders_ = {<std::cxx11::List_base<proxygen::HTTPTransaction::Chunk, std::allocatorproxygen::HTTPTransaction::Chunk >> = { M_impl = {<std::allocator<std::List_nodeproxygen::HTTPTransaction::Chunk >> = {<gnu_cxx::new_allocator<std::List_nodeproxygen::HTTPTransaction::Chunk >> = {}, }, M_node = {std::__detail::_List_node_base = {M_next = , M_prev = }, M_storage = {M_storage = "\000\000\000\000\000\000\000"}}}}, }, egressQueue = , queueHandle = , recvToAck = 43425, assocStreamId = 0, pushedTransactions = std::set with 0 elements, priority = {streamDependency = 0, exclusive = false, weight = 15 '\017'}, insertDepth = 0, currentDepth = 1, cumulativeRatio = 0, egressCalls = 0, lastResponseStatus = 200, expectedContentLengthRemaining = {storage = {{hasValue = true, {paddingForHasValue_ = {true},

First, thanks very much for releasing proxygen. It's been a fantastic building block for us. We really appreciate it. I'm running into trouble trying to use proxygen sample client code inside a binary which runs proxygen::HTTPServer.

To produce a similar error, apply the following patch:

$ git diff Makefile.am
diff --git a/proxygen/httpclient/samples/curl/Makefile.am b/proxygen/httpclient/samples/curl/Makefile
index 6cbeaab..d3024e8 100644
--- a/proxygen/httpclient/samples/curl/Makefile.am
+++ b/proxygen/httpclient/samples/curl/Makefile.am
@@ -11,7 +11,8 @@ libproxygencurl_la_SOURCES = \

 libproxygencurl_la_LIBADD = \
         ../../../lib/libproxygenlib.la \
-       ../../../lib/http/libproxygenhttp.la
+       ../../../lib/http/libproxygenhttp.la \
+       ../../../httpserver/libproxygenhttpserver.la

 libproxygencurl_la_LDFLAGS=-static

After a successful build & link (had to make clean to pickup changes), the following error:

$ ./curl
Double registration of singletons of the same underlying type; check for multiple definitions of type folly::Singleton<proxygen::SharedWheelTimer>
Aborted (core dumped)

The above exposes the symptom but isn't very useful. The symptom in my build is the following:

• If linking to libproxygenhttpserver, there's an undefined reference to proxygen::HTTPUpstreamSession
• If linking to libproxygenlib, there's an undefined reference to proxygen::HTTPServer
• If linking to both, the error with double registration of proxygen::SharedWheelTimer
• Note the proxgen .deb file does not include libproxygenhttp, so I haven't included that in my tests.

I hope that this is sensible on some level. Thank you for your attention. -dg

As per afrind's suggestion, opening an issue:

#######################################################

Is there a plan to "fold" HQ into HTTPServer proper ?

Borrowing from sample EchoServer.cpp code, would love to have something like example below.

With 0 changes to existing app, incorporate support for QUIC and later on, H3, in part related to HTTP .

################### using Protocol = HTTPServer::Protocol;

DEFINE_int32(http_port, 11000, "Port to listen on with HTTP protocol"); DEFINE_int32(spdy_port, 11001, "Port to listen on with SPDY protocol"); DEFINE_int32(h2_port, 11002, "Port to listen on with HTTP/2 protocol"); DEFINE_int32(h3_port, 11003, "Port to listen on with HTTP/3 protocol");**

... HTTPServer server(std::move(options)); server.bind(IPs);

// Start HTTPServer mainloop in a separate thread std::thread t([&] () { server.start(); });

As noted at https://reviews.facebook.net/D38373 , Proxygen does not suppress the sending of the response body when the request is a HEAD request. In fact, even if the application sends no response data, if the HTTPMessage::chunked_ flag is set, HTTP1xCodec::generateEOM() will send an empty chunk "0\r\n", which is a protocol violation.

It's arguable whether a Transfer-Encoding header should be sent on HEAD, but either way, the policy should be Proxygen's responsibility, rather than the application's, which means that the application should be allowed to call HTTPMessage::setIsChunked() on the response to a HEAD request without causing a protocol violation.

I'm not sure how much help Proxygen should give to applications wishing to implement HEAD handling. Certainly, it should not require applications to send a response body to Proxygen to be discarded, since that is inefficient, but maybe Proxygen could automatically discard a response body if one is given.

I am using proxygen and keep on getting StreamError. I have gone through ProxyGen code and I believe this error arises because there is an HTTP error (abort unacknowledged message) when sending/ or receiving. Is there a way to obtain more insight/granularity on whether the error is from a HTTP request or response?

RFC2616 is very strict when it comes to "204 No Content" responses, requiring them to actually be empty. However, not everyone follows this (that is, they return content in 204 responses), causing otherwise valid HTTP responses to be rejected.

This PR adds a CMake option ALLOW_NONEMPTY_204_RESPONSE (which sets a preprocessor macro with the same name) that disables rejecting 204 responses with content, so that you can instead ignore the body of the response in those cases.

Summary: X-link: https://github.com/facebookincubator/fizz/pull/82

X-link: https://github.com/facebookincubator/velox/pull/2487

X-link: https://github.com/facebookincubator/hsthrift/pull/101

X-link: https://github.com/facebookincubator/katran/pull/172

X-link: https://github.com/facebookincubator/mvfst/pull/273

X-link: https://github.com/fairinternal/AIRStore/pull/38

LibUnwind is a dependency of glog according to objdump. This diff adds ${LIBUNWIND_LIBRARY} to the CMake imported library glog::glog as an element in the IMPORTED_LINK_INTERFACE_LIBRARIES property.

Without this diff, there will be a linker error like this:

/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libglog.so: undefined reference to symbol '_Ux86_64_getcontext'
//usr/lib/x86_64-linux-gnu/libunwind.so.8: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

Differential Revision: D39340650

Summary: X-link: https://github.com/facebook/fb303/pull/31

Previously Folly provided a polyfill of the GflagsFlagSaver symbol, which was removed since https://github.com/facebook/folly/commit/4dadde1256f74e756510a499c86459967f78afb9. Therefore, the glog should solve the GflagsFlagSaver symbol from gflags.

This diff added gflags libraries as a dependency of the CMake project glog::glog into fbcode_builder's FindGlog.cmake, so that the fbcode_builder users will be able to automatically link with gflags.

Differential Revision: D39220438

It would be very convenient to have URL::join function to construct a new URL from an existing URL and a relative path (for example one from Location server header on 302-redirect).

Despite simplicity, implementation has some corner cases, including security sensitive ones, so it is better to have it implemented in a library than to left is an an exercise to a library user.