Experimental, scalable, high performance HTTP server

Overview

Lwan Web Server

Lwan is a high-performance & scalable web server.

The project web site contains more details.

Build status

OS Arch Release Debug Static Analysis Tests
Linux x86_64 release debug static-analysis coverity Report history tests Fuzzing Status
Linux armv7 release-arm debug-arm
FreeBSD x86_64 freebsd-release freebsd-debug
macOS x86_64 osx-release osx-debug
OpenBSD 6.6 x86_64 openbsd-release openbsd-debug openbsd-tests

Building

Before installing Lwan, ensure all dependencies are installed. All of them are common dependencies found in any GNU/Linux distribution; package names will be different, but it shouldn't be difficult to search using whatever package management tool that's used by your distribution.

Required dependencies

Optional dependencies

The build system will look for these libraries and enable/link if available.

On non-x86 systems, libucontext will be downloaded and built alongside Lwan.

Common operating system package names

Minimum to build

  • ArchLinux: pacman -S cmake zlib
  • FreeBSD: pkg install cmake pkgconf
  • Ubuntu 14+: apt-get update && apt-get install git cmake zlib1g-dev pkg-config
  • macOS: brew install cmake

Build all examples

  • ArchLinux: pacman -S cmake zlib sqlite luajit libmariadbclient gperftools valgrind
  • FreeBSD: pkg install cmake pkgconf sqlite3 lua51
  • Ubuntu 14+: apt-get update && apt-get install git cmake zlib1g-dev pkg-config lua5.1-dev libsqlite3-dev libmysqlclient-dev
  • macOS: brew install cmake mysql-connector-c sqlite [email protected] pkg-config

Build commands

Clone the repository

~$ git clone git://github.com/lpereira/lwan
~$ cd lwan

Create the build directory

~/lwan$ mkdir build
~/lwan$ cd build

Select build type

Selecting a release version (no debugging symbols, messages, enable some optimizations, etc):

~/lwan/build$ cmake .. -DCMAKE_BUILD_TYPE=Release

If you'd like to enable optimizations but still use a debugger, use this instead:

~/lwan/build$ cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo

To disable optimizations and build a more debugging-friendly version:

~/lwan/build$ cmake .. -DCMAKE_BUILD_TYPE=Debug

Build Lwan

~/lwan/build$ make

This will generate a few binaries:

  • src/bin/lwan/lwan: The main Lwan executable. May be executed with --help for guidance.
  • src/bin/testrunner/testrunner: Contains code to execute the test suite.
  • src/samples/freegeoip/freegeoip: FreeGeoIP sample implementation. Requires SQLite.
  • src/samples/techempower/techempower: Code for the TechEmpower Web Framework benchmark. Requires SQLite and MySQL libraries.
  • src/samples/clock/clock: Clock sample. Generates a GIF file that always shows the local time.
  • src/bin/tools/mimegen: Builds the extension-MIME type table. Used during build process.
  • src/bin/tools/bin2hex: Generates a C file from a binary file, suitable for use with #include.
  • src/bin/tools/configdump: Dumps a configuration file using the configuration reader API.

Remarks

Passing -DCMAKE_BUILD_TYPE=Release will enable some compiler optimizations (such as LTO) and tune the code for current architecture. Please use this version when benchmarking, as the default is the Debug build, which not only logs all requests to the standard output, but does so while holding a mutex.

The default build (i.e. not passing -DCMAKE_BUILD_TYPE=Release) will build a version suitable for debugging purposes. This version can be used under Valgrind (if its headers are present) and includes debugging messages that are stripped in the release version. Debugging messages are printed for each and every request.

On debug builds, sanitizers can be enabled. To select which one to build Lwan with, specify one of the following options to the CMake invocation line:

  • -DSANITIZER=ubsan selects the Undefined Behavior Sanitizer.
  • -DSANITIZER=address selects the Address Sanitizer.
  • -DSANITIZER=thread selects the Thread Sanitizer.

Alternative memory allocators can be selected as well. Lwan currently supports TCMalloc, mimalloc, and jemalloc out of the box. To use either one of them, pass -DALTERNATIVE_MALLOC=name to the CMake invocation line, using the names provided in the "Optional dependencies" section.

Tests

~/lwan/build$ make testsuite

This will compile the testrunner program and execute regression test suite in src/scripts/testsuite.py.

Benchmark

~/lwan/build$ make benchmark

This will compile testrunner and execute benchmark script src/scripts/benchmark.py.

Coverage

Lwan can also be built with the Coverage build type by specifying -DCMAKE_BUILD_TYPE=Coverage. This enables the generate-coverage make target, which will run testrunner to prepare a test coverage report with lcov.

Every commit in this repository triggers the generation of this report, and results are publicly available.

Running

Set up the server by editing the provided lwan.conf; the format is explained in details below. (Lwan will try to find a configuration file based in the executable name in the current directory; testrunner.conf will be used for the testrunner binary, lwan.conf for the lwan binary, and so on.)

Configuration files are loaded from the current directory. If no changes are made to this file, running Lwan will serve static files located in the ./wwwroot directory. Lwan will listen on port 8080 on all interfaces.

Lwan will detect the number of CPUs, will increase the maximum number of open file descriptors and generally try its best to autodetect reasonable settings for the environment it's running on. Many of these settings can be tweaked in the configuration file, but it's usually a good idea to not mess with them.

Optionally, the lwan binary can be used for one-shot static file serving without any configuration file. Run it with --help for help on that.

Configuration File

Format

Lwan uses a familiar key = value configuration file syntax. Comments are supported with the # character (similar to e.g. shell scripts, Python, and Perl). Nested sections can be created with curly brackets. Sections can be empty; in this case, curly brackets are optional.

some_key_name is equivalent to some key name in configuration files (as an implementation detail, code reading configuration options will only be given the version with underscores).

Values can contain environment variables. Use the syntax ${VARIABLE_NAME}. Default values can be specified with a colon (e.g. ${VARIABLE_NAME:foo}, which evaluates to ${VARIABLE_NAME} if it's set, or foo otherwise).

sound volume = 11 # This one is 1 louder

playlist metal {
   files = '''
	/multi/line/strings/are/supported.mp3
	/anything/inside/these/are/stored/verbatim.mp3
   '''
}

playlist chiptune {
   files = """
	/if/it/starts/with/single/quotes/it/ends/with/single/quotes.mod
	/but/it/can/use/double/quotes.s3m
   """
}

Some examples can be found in lwan.conf and techempower.conf.

Value types

Type Description
str Any kind of free-form text, usually application specific
int Integer number. Range is application specific
time Time interval. See table below for units
bool Boolean value. See table below for valid values

Time Intervals

Time fields can be specified using multipliers. Multiple can be specified, they're just added together; for instance, "1M 1w" specifies "1 month and 1 week" (37 days). The following table lists all known multipliers:

Multiplier Description
s Seconds
m Minutes
h Hours
d Days
w 7-day Weeks
M 30-day Months
y 365-day Years

A number with a multiplier not in this table is ignored; a warning is issued while reading the configuration file. No spaces must exist between the number and its multiplier.

Boolean Values

True Values False Values
Any integer number different than 0 0
on off
true false
yes no

Global Settings

It's generally a good idea to let Lwan decide the best settings for your environment. However, not every environment is the same, and not all uses can be decided automatically, so some configuration options are provided.

Option Type Default Description
keep_alive_timeout time 15 Timeout to keep a connection alive
quiet bool false Set to true to not print any debugging messages. Only effective in release builds.
reuse_port bool false Sets SO_REUSEPORT to 1 in the master socket
expires time 1M 1w Value of the "Expires" header. Default is 1 month and 1 week
threads int 0 Number of I/O threads. Default (0) is the number of online CPUs
proxy_protocol bool false Enables the PROXY protocol. Versions 1 and 2 are supported. Only enable this setting if using Lwan behind a proxy, and the proxy supports this protocol; otherwise, this allows anybody to spoof origin IP addresses
max_post_data_size int 40960 Sets the maximum number of data size for POST requests, in bytes

Straitjacket

Lwan can drop its privileges to a user in the system, and limit its filesystem view with a chroot. While not bulletproof, this provides a first layer of security in the case there's a bug in Lwan.

In order to use this feature, declare a straitjacket section, and set some options. This requires Lwan to be executed as root.

Although this section can be written anywhere in the file (as long as it is a top level declaration), if any directories are open, due to e.g. instantiating the serve_files module, Lwan will refuse to start. (This check is only performed on Linux as a safeguard for malconfiguration.)

Option Type Default Description
user str NULL Drop privileges to this user name
chroot str NULL Path to chroot()
drop_capabilities bool true Drop all capabilities with capset(2) (under Linux), or pledge(2) (under OpenBSD).

Headers

If there's a need to specify custom headers for each response, one can declare a headers section in the global scope. The order which this section appears isn't important.

For example, this declaration:

headers {
	Server = Apache/1.0.0 or nginx/1.0.0 (at your option)
	Some-Custom-Header = ${WITH_THIS_ENVIRONMENT_VARIABLE}
}

Will both override the Server header (Server: lwan won't be sent), and set Some-Custom-Header with the value obtained from the environment variable $WITH_THIS_ENVIRONMENT_VARIABLE.

Some headers can't be overridden, as that would cause issues when sending their actual values while servicing requests. These include but is not limited to:

  • Date
  • Expires
  • WWW-Authenticate
  • Connection
  • Content-Type
  • Transfer-Encoding
  • All Access-Control-Allow- headers

Header names are also case-insensitive (and case-preserving). Overriding SeRVeR will override the Server header, but send it the way it was written in the configuration file.

Listeners

In order to specify which interfaces Lwan should listen on, a listener section must be specified. Only one listener per Lwan process is accepted at the moment. The only parameter to a listener block is the interface address and the port to listen on; anything inside a listener section are instances of modules.

The syntax for the listener parameter is ${ADDRESS}:${PORT}, where ${ADDRESS} can either be * (binding to all interfaces), an IPv6 address (if surrounded by square brackets), an IPv4 address, or a hostname. If systemd's socket activation is used, systemd can be specified as a parameter.

Routing URLs Using Modules or Handlers

In order to route URLs, Lwan matches the largest common prefix from the request URI with a set of prefixes specified in the listener section. How a request to a particular prefix will be handled depends on which handler or module has been declared in the listener section. Handlers and modules are similar internally; handlers are merely functions and hold no state, and modules holds state (named instance). Multiple instances of a module can appear in a listener section.

There is no special syntax to attach a prefix to a handler or module; all the configuration parser rules apply here. Use ${NAME} ${PREFIX} to link the ${PREFIX} prefix path to either a handler named ${NAME} (if ${NAME} begins with &, as with C's "address of" operator), or a module named ${NAME}. Empty sections can be used here.

Each module will have its specific set of options, and they're listed in the next sections. In addition to configuration options, a special authorization section can be present in the declaration of a module instance. Handlers do not take any configuration options, but may include the authorization section.

A list of built-in modules can be obtained by executing Lwan with the -m command-line argument. The following is some basic documentation for the modules shipped with Lwan.

File Serving

The serve_files module will serve static files, and automatically create directory indices or serve pre-compressed files. It'll generally try its best to serve files in the fastest way possible according to some heuristics.

Option Type Default Description
path str NULL Path to a directory containing files to be served
index_path str index.html File name to serve as an index for a directory
serve_precompressed_path bool true If $FILE.gz exists, is smaller and newer than $FILE, and the client accepts gzip encoding, transfer it
auto_index bool true Generate a directory list automatically if no index_path file present. Otherwise, yields 404
auto_index_readme bool true Includes the contents of README files as part of the automatically generated directory index
directory_list_template str NULL Path to a Mustache template for the directory list; by default, use an internal template
read_ahead int 131702 Maximum amount of bytes to read ahead when caching open files. A value of 0 disables readahead. Readahead is performed by a low priority thread to not block the I/O threads while file extents are being read from the filesystem.
cache_for time 5s Time to keep file metadata (size, compressed contents, open file descriptor, etc.) in cache

Lua

The lua module will allow requests to be serviced by scripts written in the Lua programming language. Although the functionality provided by this module is quite spartan, it's able to run frameworks such as Sailor.

Scripts can be served from files or embedded in the configuration file, and the results of loading them, the standard Lua modules, and (optionally, if using LuaJIT) optimizing the code will be cached for a while. Each I/O thread in Lwan will create an instance of a Lua VM (i.e. one lua_State struct for every I/O thread), and each Lwan coroutine will spawn a Lua thread (with lua_newthread()) per request. Because of this, Lua scripts can't use global variables, as they may be not only serviced by different threads, but the state will be available only for the amount of time specified in the cache_period configuration option.

There's no need to have one instance of the Lua module for each endpoint; a single script, embedded in the configuration file or otherwise, can service many different endpoints. Scripts are supposed to implement functions with the following signature: handle_${METHOD}_${ENDPOINT}(req), where ${METHOD} can be a HTTP method (i.e. get, post, head, etc.), and ${ENDPOINT} is the desired endpoint to be handled by that function. The special ${ENDPOINT} root can be specified to act as a catchall. The req parameter points to a metatable that contains methods to obtain information from the request, or to set the response, as seen below:

  • req:query_param(param) returns the query parameter (from the query string) with the key param, or nil if not found
  • req:post_param(param) returns the post parameter (only for ${POST} handlers) with the key param, or nil if not found
  • req:set_response(str) sets the response to the string str
  • req:say(str) sends a response chunk (using chunked encoding in HTTP)
  • req:send_event(event, str) sends an event (using server-sent events)
  • req:cookie(param) returns the cookie named param, or nil is not found
  • req:set_headers(tbl) sets the response headers from the table tbl; a header may be specified multiple times by using a table, rather than a string, in the table value ({'foo'={'bar', 'baz'}}); must be called before sending any response with say() or send_event()
  • req:sleep(ms) pauses the current handler for the specified amount of milliseconds
  • req:ws_upgrade() returns 1 if the connection could be upgraded to a WebSocket; 0 otherwise
  • req:ws_write(str) sends str through the WebSocket-upgraded connection
  • req:ws_read() returns a string with the contents of the last WebSocket frame, or a number indicating an status (ENOTCONN/107 on Linux if it has been disconnected; EAGAIN/11 on Linux if nothing was available; ENOMSG/42 on Linux otherwise). The return value here might change in the future for something more Lua-like.

Handler functions may return either nil (in which case, a 200 OK response is generated), or a number matching an HTTP status code. Attempting to return an invalid HTTP status code or anything other than a number or nil will result in a 500 Internal Server Error response being thrown.

Option Type Default Description
default_type str text/plain Default MIME-Type for responses
script_file str NULL Path to Lua script
cache_period time 15s Time to keep Lua state loaded in memory
script str NULL Inline lua script

Rewrite

The rewrite module will match patterns in URLs and give the option to either redirect to another URL, or rewrite the request in a way that Lwan will handle the request as if it were made in that way originally.

Forked from Lua 5.3.1, the regular expresion engine may not be as feature-packed as most general-purpose engines, but has been chosen specifically because it is a deterministic finite automaton in an attempt to make some kinds of denial of service attacks not possible.

The new URL can be specified using a simple text substitution syntax, or use Lua scripts; Lua scripts will contain the same metamethods available in the req metatable provided by the Lua module, so it can be quite powerful.

Each instance of the rewrite module will require a pattern and the action to execute when such pattern is matched. Patterns are evaluated in the order they appear in the configuration file, and are specified using nested sections in the configuration file. For instance, consider the following example, where two patterns are specified:

rewrite /some/base/endpoint {
    pattern posts/(%d+) {
        # Matches /some/base/endpointposts/2600 and /some/base/endpoint/posts/2600
        rewrite_as = /cms/view-post?id=%1
    }
    pattern imgur/(%a+)/(%g+) {
        # Matches /some/base/endpointimgur/gif/mpT94Ld and /some/base/endpoint/imgur/gif/mpT94Ld
        redirect_to = https://i.imgur.com/%2.%1
    }
}

This example defines two patterns, one providing a nicer URL that's hidden from the user, and another providing a different way to obtain a direct link to an image hosted on a popular image hosting service (i.e. requesting /some/base/endpoint/imgur/mp4/4kOZNYX will redirect directly to a resource in the Imgur service).

The value of rewrite_as or redirect_to can be Lua scripts as well; in which case, the option expand_with_lua must be set to true, and, instead of using the simple text substitution syntax as the example above, a function named handle_rewrite(req, captures) has to be defined instead. The req parameter is documented in the Lua module section; the captures parameter is a table containing all the captures, in order. This function returns the new URL to redirect to.

This module has no options by itself. Options are specified in each and every pattern.

Option Type Default Description
rewrite_as str NULL Rewrite the URL following this pattern
redirect_to str NULL Redirect to a new URL following this pattern
expand_with_lua bool false Use Lua scripts to redirect to or rewrite a request

redirect_to and rewrite_as options are mutually exclusive, and one of them must be specified at least.

Redirect

The redirect module will, as it says in the tin, generate a 301 Moved permanently (by default; the code can be changed, see below) response, according to the options specified in its configuration. Generally, the rewrite module should be used instead as it packs more features; however, this module serves also as an example of how to write Lwan modules (less than 100 lines of code).

If the to option is not specified, it always generates a 500 Internal Server Error response. Specifying an invalid HTTP code, or a code that Lwan doesn't know about (see enum lwan_http_status), will produce a 301 Moved Permanently response.

Option Type Default Description
to str NULL The location to redirect to
code int 301 The HTTP code to perform a redirect

Response

The response module will generate an artificial response of any HTTP code. In addition to also serving as an example of how to write a Lwan module, it can be used to carve out voids from other modules (e.g. generating a 405 Not Allowed response for files in /.git, if / is served with the serve_files module).

If the supplied code falls outside the response codes known by Lwan, a 404 Not Found error will be sent instead.

Option Type Default Description
code int 999 A HTTP response code

Authorization Section

Authorization sections can be declared in any module instance or handler, and provides a way to authorize the fulfillment of that request through the standard HTTP authorization mechanism. In order to require authorization to access a certain module instance or handler, declare an authorization section with a basic parameter, and set one of its options.

Option Type Default Description
realm str Lwan Realm for authorization. This is usually shown in the user/password UI in browsers
password_file str NULL Path for a file containing username and passwords (in clear text). The file format is the same as the configuration file format used by Lwan

Hacking

Please read this section (and follow it) if you're planning on contributing to Lwan. There's nothing unexpected here; this mostly follows the rules and expectations of many other FOSS projects, but every one expects things a little bit different from one another.

Coding Style

Lwan tries to follow a consistent coding style throughout the project. If you're considering contributing a patch to the project, please respect this style by trying to match the style of the surrounding code. In general:

  • global_variables_are_named_like_this, even though they tend to be rare and should be marked as static (with rare exceptions)
  • Local variables are usually shorter, e.g. local_var, i, conn
  • Struct names are often as short as they're descriptive. typedef for structs are rarely used in Lwan
  • Header files should use #pragma once instead of the usual include guard hackery
  • Functions that are used between .c files but are not APIs to be exposed to liblwan should have their prototype added to lwan-private.h
  • Functions should be short and sweet. Exceptions may apply
  • Public functions should be prefixed with lwan_
  • Public types should be prefixed with lwan_
  • Private functions must be static, and can be named without the lwan_ prefix
  • Code is indented with 4 spaces; don't use tabs
  • There's a suggested line break at column 80, but it's not enforced
  • /* Old C-style comments are preferred */
  • clang-format can be used to format the source code in an acceptable way; a .clang-format file is provided

Tests

If modifying well-tested areas of the code (e.g. the event loop, HTTP parser, etc.), please add a new integration test and make sure that, before you send a pull request, all tests (including the new ones you've sent) are working. Tests can be added by modifying src/scripts/testsuite.py, and executed by either invoking that script directly from the source root, or executing the testsuite build target.

Some tests will only work on Linux, and won't be executed on other platforms.

Fuzz-testing

Lwan is automatically fuzz-tested by OSS-Fuzz. To fuzz-test locally, though, one can follow the instructions to test locally.

This fuzzes only the request parsing code. There are plans to add fuzzing drivers for other parts of the code, including the rewriting engine, configuration file reader, template parser, and URL routing.

Exporting APIs

The shared object version of liblwan on ELF targets (e.g. Linux) will use a symbol filter script to hide symbols that are considered private to the library. Please edit src/lib/liblwan.sym to add new symbols that should be exported to liblwan.so.

Using Git and Pull Requests

Lwan tries to maintain a source history that's as flat as possible, devoid of merge commits. This means that pull requests should be rebased on top of the current master before they can be merged; sometimes this can be done automatically by the GitHub interface, sometimes they need some manual work to fix conflicts. It is appreciated if the contributor fixes these conflicts when asked.

It is advisable to push your changes to your fork on a branch-per-pull request, rather than pushing to the master branch; the reason is explained below.

Please ensure that Git is configured properly with your name (it doesn't really matter if it is your legal name or a nickname, but it should be enough to credit you) and a valid email address. There's no need to add Signed-off-by lines, even though it's fine to send commits with them.

If a change is requested in a pull request, you have two choices:

  • Reply asking for clarification. Maybe the intentions were not clear enough, and whoever asked for changes didn't fully understand what you were trying to achieve
  • Fix the issue. When fixing issues found in pull requests, please use interactive rebases to squash or fixup commits; don't add your fixes on top of your tree. Do not create another pull request just to accomodate the changes. After rewriting the history locally, force-push to your PR branch; the PR will update automatically with your changes. Rewriting the history of development branches is fine, and force-pushing them is normal and expected

It is not enforced, but it is recommended to create smaller commits. How commits are split in Lwan is pretty much arbitrary, so please take a look at the commit history to get an idea on how the division should be made. Git offers a plethora of commands to achieve this result: the already mentioned interactive rebase, the -p option to git add, and git commit --amend are good examples.

Commit messages should have one line of summary (~72 chars), followed by an empty line, followed by paragraphs of 80-char lines explaining the change. The paragraphs explaining the changes are usually not necessary if the summary is good enough. Try to write good commit messages.

Licensing

Lwan is licensed under the GNU General Public License, version 2, or (at your option), any later version. Therefore:

  • Code must be either LGPLv2.1, GPLv2, a permissive "copyfree" license that is compatible with GPLv2 (e.g. MIT, BSD 3-clause), or public domain code (e.g. CC0)
  • Although the program can be distributed and used as if it were licensed as GPLv3, its code must be compatible with GPLv2 as well; no new code can be licensed under versions of GPL newer than 2
  • Likewise, code licensed under licenses compatible with GPLv3 but incompatible with GPLv2 (e.g. Apache 2) are not suitable for inclusion in Lwan
  • Even if the license does not specify that credit should be given (e.g. CC0-licensed code), please give credit to the original author for that piece of code
  • Contrary to popular belief, it is possible to use a GPL'd piece of code on a server without having to share the code for your application. It is only when the binary of that server is shared that source must be available to whoever has that binary. Merely accessing a Lwan server through HTTP does not qualify as having access to the binary program that's running on the server
  • When in doubt, don't take legal advice from a README file: please consult a lawyer that understands free software licensing

Portability

While Lwan was written originally for Linux, it has been ported to BSD systems as well. The build system will detect the supported features and build support library functions as appropriate.

For instance, epoll has been implemented on top of kqueue, and Linux-only syscalls and GNU extensions have been implemented for the supported systems. This blog post explains the details and how #include_next is used.

Performance

It can achieve good performance, yielding about 320000 requests/second on a Core i7 laptop for requests without disk access, and without pipelining.

When disk I/O is required, for files up to 16KiB, it yields about 290000 requests/second; for larger files, this drops to 185000 requests/second, which isn't too shabby either.

These results, of course, with keep-alive connections, and with weighttp running on the same machine (and thus using resources that could be used for the webserver itself).

Without keep-alive, these numbers drop around 6-fold.

IRC Channel

There is an IRC channel (#lwan) on Freenode. A standard IRC client can be used. A web IRC gateway is also available.

Lwan in the wild

Here's a non-definitive list of third-party stuff that uses Lwan and have been seen in the wild. Help build this list!

Some other distribution channels were made available as well:

Lwan has been also used as a benchmark:

Mentions in academic journals:

Some talks mentioning Lwan:

Not really third-party, but alas:

Lwan quotes

These are some of the quotes found in the wild about Lwan. They're presented in no particular order. Contributions are appreciated:

"I read lwan's source code. Especially, the part of using coroutine was very impressive and it was more interesting than a good novel. Thank you for that." -- @patagonia

"For the server side, we're using Lwan, which can handle 100k+ reqs/s. It's supposed to be super robust and it's working well for us." -- @fawadkhaliq

"Insane C thing" -- Michael Sproul

"I've never had a chance to thank you for Lwan. It inspired me a lot to develop Zewo" -- @paulofariarl

"Let me say that lwan is a thing of beauty. I got sucked into reading the source code for pure entertainment, it's so good. high five" -- @kwilczynski

"Nice work with Lwan! I haven't looked that carefully yet but so far I like what I saw. You definitely have the right ideas." -- @thinkingfish

"Lwan is a work of art. Every time I read through it, I am almost always awe-struck." -- @neurodrone

"For Round 10, Lwan has taken the crown" -- TechEmpower

"Jeez this is amazing. Just end to end, rock solid engineering. (...) But that sells this work short." kjeetgill

"I am only a spare time C coder myself and was surprised that I can follow the code. Nice!" cntlzw

"Impressive all and all, even more for being written in (grokkable!) C. Nice work." tpaschalis

Comments
  • Segmentation fault: 11

    Segmentation fault: 11

    After building from source, I got this error:

    [16:30PM] sandbox$ /Users/jeremywoertink/Development/lwan/build/lwan/lwan -r ./
    Serving files from ./.
    Using 8 threads, maximum 1280 sockets per thread.
    Listening on http://0.0.0.0:8080.
    Ready to serve.
    Segmentation fault: 11
    

    I'm running on macOS 10.12.3 built with cmake 3.7.0 and running the default LLVM 8.0.0.

    opened by jwoertink 29
  • Error when response large content using strbuf_set_static

    Error when response large content using strbuf_set_static

    We are working on a lwan module that read images from an external service (written using Apache thrift). The specify content is images that user upload, then save to thrift backend, response to web's user by lwan.

    Every object can be served fine when its size is smaller than 2.4MB. But when it's larger that than, lwan response with content size = 2.4 MB, even when we sent content length > 2.4M, that's cause content-length mis-match, image received is not enough (it was cut) to 2.4MB and client have to wait until timeout. response->mime_type = lwan_determine_mime_type_for_file_name(file_name); strbuf_set(response->buffer,(char*) fi->metadata.content, fi->metadata.size);

    when metadata.content has size larger than 2.4MB, problem occurs. Is this a known issue? I tested when lwan serve files module and it works fine.

    opened by whatvn 29
  • Release tags for lwan

    Release tags for lwan

    I want to maintain lwan port for FreeBSD, but is has to use git do download fixed commits to download source and apply patches to.

    Could you create tags/releases to make versioning much easer.

    opened by eirnym 26
  • Respond very slowed when POST body large than 1024

    Respond very slowed when POST body large than 1024

    For example, the following command is very fast:

    curl -i -X POST "http://172.30.8.82:8080/world" -d "appname=foo&appkey=bar"
    

    But, if I increase the POST body size large than 1024, it is very solow. For example:

    curl -i -X POST "http://localhost:8080/world" -d "appname=foo&appkey=bar&algo=1234567890123456789012345678901234567890&prikey=MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAMci8EEQm6kRyIXmD3/ckp98JZjS4QT2CXAVygr4i6EaZhm6pFQlWSujAuuG9QnbskGf3bcKjdysPs622wmtrJR6Bl330JxmJixS+FJyi18J7G6x99pn4NiExiaNlQXM7WMKh1308AYRK4qqfkwTLgYaGo7GgqVoXIS2tgYw+nWPAgMBAAECgYBWPuuPQoIoV2thkqJj3kQijsI5zhtm6qhrwwQefEb9H1HZEtzZFNPp8k/xciHugxhRE8wS2p9WU8NgBBIZvv5fDGaLbHAO2cEtJoO06S4mQKw6ZfRvrUJaNVp5mNasjwA8QfXhawmr3zF13MK4D5qwmc5UqrnU03OAI2YQ+F/tAQJBAOIe+rWQV+Zh2Fv3415WqMe0rstwIohpHYjefGn+R3fM+d6haRxodt6ZTfNWpwkgf4xPKWq8jwCSvr8WJkxZy50CQQDhcyZ6kka4MMjIq+lVsYrVPm6i3GAegutB2fcGSsodTxCMIas9u5d8xNfcc/B9nh533po8wuq0e/WrqcrWnywbAkA02DCXRgiE2tEJjUEhxvbi1SMUgDxRzN8OIZDJ5JcnNEwg72/1J6TAGOvStqmKxIGKdOWlADQbWjAh0DdbGJExAkEA2kKXkvyAGJuqFpRP7lVWugGaLHM7Xu6SAYuZ2e6X424C8Bc8aCRvkSwkatmXjpWAm73XBlIR1S9Y9JsJs2Qf1wJBAIirbU5JXVedimYaahxGRNdOFlj0sdUOrXa8sEYrGrTcRZjxpcP+ph0XdpzhexN2fQc2vllP/wDDT5EowN7fZjo=&data=MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTIzNDU2Nzg5MGFiY2RlZmc="
    

    I try to modify the DEFAULT_BUFFER_SIZE and CORO_BUMP_PTR_ALLOC_SIZE, but it didn't work.

    opened by fangxinmiao 14
  • SIGSEGV: coro_reset:getcontext() on uClibc armv7l

    SIGSEGV: coro_reset:getcontext() on uClibc armv7l

    Hello,

    I build lwan-latest using ImportExecutables.cmake from a native build, using a buildroot uClibc (arm-buildroot-linux-uclibcgnueabihf).

    cd ../amd64/src/
    make clean; rm -rf CMakeCache.txt CMakeFiles ImportExecutables.cmake Makefile cmake_install.cmake empty.c lwan-build-config.h lwan.pc && cmake -DCMAKE_BUILD_TYPE=Debug .. && make
    
    cd ../arm/src/
    make clean; rm -rf CMakeCache.txt CMakeFiles ImportExecutables.cmake Makefile cmake_install.cmake empty.c lwan-build-config.h lwan.pc && cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=/my/buildroot/arm-buildroot-linux-uclibcgnueabihf_sdk-buildroot/share/buildroot/toolchainfile.cmake -DIMPORT_EXECUTABLES=$PWD/../amd64/src/ImportExecutables.cmake -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_C_FLAGS='-g -O0' .. && make && scp src/bin/lwan/lwan target:/sbin/lwan
    

    I then run lwan (under gdb) and make a request to it to trigger the problem.

    # gdb lwan
    Reading symbols from /sbin/lwan...done.
    (gdb) b lib/lwan-coro.c:294
    Breakpoint 1 at 0x189e8: file src/lib/lwan-coro.c, line 294.
    (gdb) run
    Starting program: /sbin/lwan 
    9300 lwan-job.c:104 lwan_job_thread_init() Initializing low priority job thread
    9300 lwan-tables.c:47 lwan_tables_init() Uncompressing MIME type table: 8082->31348 bytes, 953 entries
    9300 lwan.c:495 setup_from_config() Loading configuration file: lwan.conf
    9300 lwan.c:365 parse_listener_prefix() Initializing module serve_files from config
    9300 lwan-response.c:77 lwan_response_init() Initializing default response
    9300 lwan.c:696 lwan_init_with_config() Initializing lwan web server
    9300 lwan.c:723 lwan_init_with_config() Using 1 threads, maximum 4096 sockets per thread
    9300 lwan-readahead.c:177 lwan_readahead_init() Initializing low priority readahead thread
    9300 lwan-thread.c:701 lwan_thread_init() Initializing threads
    9300 lwan-thread.c:710 lwan_thread_init() Pending client file descriptor queue has 256 items
    9305 lwan-thread.c:453 thread_io_loop() Worker thread #1 starting
    9300 lwan-thread.c:743 lwan_thread_init() Worker threads created and ready to serve
    9300 lwan-socket.c:247 lwan_socket_init() Initializing sockets
    9300 lwan-socket.c:236 setup_socket_normally() a
    9300 lwan-socket.c:168 listen_addrinfo() Listening on http://0.0.0.0:8080
    9300 lwan-socket.c:239 setup_socket_normally() b
    9300 lwan.c:831 lwan_main_loop() Ready to serve
    [New LWP 9305]
    [Switching to LWP 9305]
    
    Breakpoint 1, coro_reset (coro=0x4f200, func=0x254f9 <process_request_coro>, data=0x50c58) at src/lwan2/src/lib/lwan-coro.c:294
    294	    lwan_status_info("getcontext <");
    (gdb) l
    289	    coro->context[5 /* EIP */] = (uintptr_t)coro_entry_point;
    290	
    291	#define STACK_PTR 6
    292	    coro->context[STACK_PTR] = (uintptr_t)stack;
    293	#else
    294	    lwan_status_info("getcontext <");
    295	    getcontext(&coro->context);
    296	    lwan_status_info("getcontext >");
    297	
    298	    coro->context.uc_stack.ss_sp = stack;
    (gdb) p stack
    $1 = (unsigned char *) 0x76f0f000 ""
    (gdb) p coro->context 
    $2 = {uc_flags = 0, uc_link = 0x0, uc_stack = {ss_sp = 0x0, ss_flags = 0, ss_size = 0}, uc_mcontext = {trap_no = 0, error_code = 0, oldmask = 0, arm_r0 = 0, 
        arm_r1 = 0, arm_r2 = 0, arm_r3 = 0, arm_r4 = 0, arm_r5 = 0, arm_r6 = 0, arm_r7 = 0, arm_r8 = 0, arm_r9 = 0, arm_r10 = 0, arm_fp = 0, arm_ip = 0, arm_sp = 0, 
        arm_lr = 0, arm_pc = 0, arm_cpsr = 0, fault_address = 0}, uc_sigmask = {__val = {0, 0}}, uc_regspace = {0, 0, 0, 0, 0, 0, 325168, 1, 4, 
        0 <repeats 39 times>, 325272, 1, 4, 0 <repeats 77 times>}}
    (gdb) c
    Continuing.
    9305 lwan-coro.c:294 coro_reset() getcontext <
    Program received signal SIGSEGV, Segmentation fault.
    0x76e79810 in getcontext () from /lib//libc.so.0
    (gdb) quit
    

    I am not quite sure how to debug this any further. Please let me know if I can help with more output.

    opened by dset0x 13
  • Custom Paths

    Custom Paths

    Hi Ipereira, how does one go about writing custom paths such as /hello/John and getting John as a param, and different paths leading to different handlers ?

    Thank You

    opened by GuacheSuede 12
  • Failure to build for ARM

    Failure to build for ARM

    Hello!

    Great work with this library! I'm trying to build it for ARM and following your README.md steps, I tried this:

    cmake -DZLIB_ROOT=/mnt/c/Dev/Linux/zlib -DCMAKE_BUILD_TYPE=Release ..
    make
    

    The cmake works properly, it find my custom Zlib cross compiled to my device. However, when we call make it say that it can't find it:

    guto@GUTO-SURFACE3:/mnt/c/Dev/Linux/lwan/build$ make
    Scanning dependencies of target mimegen
    [  2%] Creating directories for 'mimegen'
    [  4%] No download step for 'mimegen'
    [  7%] No patch step for 'mimegen'
    [  9%] No update step for 'mimegen'
    [ 11%] Performing configure step for 'mimegen'
    -- The C compiler identification is GNU 5.2.0
    -- Check for working C compiler: /mnt/c/Dev/Linux/toolchain/bin/arm-arm1176jzs-linux-gnueabi-gcc
    -- Check for working C compiler: /mnt/c/Dev/Linux/toolchain/bin/arm-arm1176jzs-linux-gnueabi-gcc -- works
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- The CXX compiler identification is GNU 5.2.0
    -- Check for working CXX compiler: /mnt/c/Dev/Linux/toolchain/bin/arm-arm1176jzs-linux-gnueabi-g++
    -- Check for working CXX compiler: /mnt/c/Dev/Linux/toolchain/bin/arm-arm1176jzs-linux-gnueabi-g++ -- works
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    CMake Error at /usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:108 (message):
      Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR)
    Call Stack (most recent call first):
      /usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:315 (_FPHSA_FAILURE_MESSAGE)
      /usr/share/cmake-2.8/Modules/FindZLIB.cmake:85 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
      CMakeLists.txt:6 (find_package)
    
    
    -- Configuring incomplete, errors occurred!
    See also "/mnt/c/Dev/Linux/lwan/build/common/tools/src/mimegen-build/CMakeFiles/CMakeOutput.log".
    make[2]: *** [common/tools/src/mimegen-stamp/mimegen-configure] Error 1
    make[1]: *** [common/CMakeFiles/mimegen.dir/all] Error 2
    make: *** [all] Error 2
    

    Can someone give a hand with it?

    Thank you!

    opened by galvesribeiro 12
  • Illegal instruction (core dumped) in latest docker image

    Illegal instruction (core dumped) in latest docker image

    i am using docker run --rm -P -v $(pwd):/html ghcr.io/lpereira/lwan -r /html

    its giving this error Ekran Resmi 2022-05-09 18 43 03

    and inside of lwan.conf

    # Timeout in seconds to keep a connection alive.
    keep_alive_timeout = 15
    
    # Set to true to not print any debugging messages. (Only effective in
    # release builds.)
    quiet = false
    
    # Value of "Expires" header. Default is 1 month and 1 week.
    expires = 1M 1w
    
    # Number of I/O threads. Default (0) is number of online CPUs.
    threads = 0
    
    # Disable HAProxy's PROXY protocol by default. Only enable if needed.
    proxy_protocol = false
    
    # Enable straitjacket by default. The `drop_capabilities` option is `true`
    # by default.  Other options may require more privileges.
    straitjacket
    
    listener *:8080 {
        serve_files / {
                path = ./wwwroot
    
                # When requesting for file.ext, look for a smaller/newer file.ext.gz,
                # and serve that instead if `Accept-Encoding: gzip` is in the
                # request headers.
                serve precompressed files = true
        }
    }
    

    basically, i want to serve my static files with lwan. i do not edit conf file, its coming from original docker image

    opened by haliliceylan 11
  • Could not drop capabilities: Invalid argument (error number 22)

    Could not drop capabilities: Invalid argument (error number 22)

    I ran into this error after build and load with original lwan.conf. Did I miss something?

    localhost:/briteFaaS/runtime/lwan# ./build/src/bin/testrunner/testrunner Loading configuration file: testrunner.conf Could not drop capabilities: Invalid argument (error number 22)

    opened by petermp79 11
  • epoll is not available on mac

    epoll is not available on mac

    epoll() does not exist on Mac OS X. It does have the somewhat-similar kqueue system call. Alternatively, you can use a library such as libevent that abstracts away the difference. So,is there anything I can do on Mac?

    opened by tkisme 11
  • Unknown type name 'lwan_t'

    Unknown type name 'lwan_t'

    How to compile helloworld.c?? also trying to compile with pkg-config , there is an error stdc-predef.h fatal error /lwan-build-config.h no such file or directory. Though it is there in usr local include lwan What include ausserdem lwan.h?

    opened by Globik 10
  • 12% of cycles spent percent-decoding URLs

    12% of cycles spent percent-decoding URLs

    Replacing url_decode() with strlen() increases Lwan throughput by about 12% when servicing Hello, World responses. The current algorithm is not only very branchy, it also essentially does a byte-by-byte copy of every character that comes into the function, even if no decoding was necessary for that particular string.

    opened by lpereira 2
  • Unexpected socket closing, round#2

    Unexpected socket closing, round#2

    Hi,

    I found an other weird behaviour in Lwan when I started to use our server for serving video streams for masses. After a point, when the client drop the connection, for example it had a network issue, it's connection stuck in the queue and start closing its socket after that connection was dropped.

    Steps for reproducing:

    1. Please apply this patch:
    diff --git a/src/lib/lwan-thread.c b/src/lib/lwan-thread.c
    index 797acf98..db242a73 100644
    --- a/src/lib/lwan-thread.c
    +++ b/src/lib/lwan-thread.c
    @@ -940,6 +940,7 @@ static void *thread_io_loop(void *data)
    
                 if (UNLIKELY(event->events & (EPOLLRDHUP | EPOLLHUP))) {
                     if ((conn->flags & CONN_AWAITED_FD) != CONN_SUSPENDED) {
    +                    lwan_status_info("HUP fd: %d flags: %x, %d", lwan_connection_get_fd(lwan, conn), conn->flags, (conn->flags & CONN_AWAITED_FD) != CONN_SUSPENDED);
                         timeout_queue_expire(&tq, conn);
                         continue;
                     }
    diff --git a/src/lib/lwan-tq.c b/src/lib/lwan-tq.c
    index 102443c1..f4e36a8d 100644
    --- a/src/lib/lwan-tq.c
    +++ b/src/lib/lwan-tq.c
    @@ -94,7 +94,9 @@ void timeout_queue_expire(struct timeout_queue *tq,
             conn->coro = NULL;
         }
    
    -    close(lwan_connection_get_fd(tq->lwan, conn));
    +    if (close(lwan_connection_get_fd(tq->lwan, conn))) {
    +        lwan_status_info("close error, fd: %d", lwan_connection_get_fd(tq->lwan, conn));
    +    }
     }
    
     void timeout_queue_expire_waiting(struct timeout_queue *tq)
    diff --git a/src/lib/lwan.c b/src/lib/lwan.c
    index 9fd66e68..0a7db006 100644
    --- a/src/lib/lwan.c
    +++ b/src/lib/lwan.c
    @@ -54,7 +54,7 @@ clockid_t monotonic_clock_id = CLOCK_MONOTONIC;
    
     static const struct lwan_config default_config = {
         .listener = "localhost:8080",
    -    .keep_alive_timeout = 15,
    +    .keep_alive_timeout = 2,
         .quiet = false,
         .proxy_protocol = false,
         .allow_cors = false,
    diff --git a/src/samples/hello/main.c b/src/samples/hello/main.c
    index 16e68bea..b7dc6720 100644
    --- a/src/samples/hello/main.c
    +++ b/src/samples/hello/main.c
    @@ -26,6 +26,9 @@ LWAN_HANDLER_ROUTE(hello_world, "/")
         response->mime_type = "text/plain";
         lwan_strbuf_set_static(response->buffer, message, sizeof(message) - 1);
    
    +    while (1)
    +        lwan_request_sleep(request, 10);
    +
         return HTTP_OK;
     }
    
    
    1. Compile Lwan and please start hello sample app.
    2. Please compile and run this small test application:
    
    #include <stdlib.h>
    #include <unistd.h>
    #include <sys/types.h>
    #include <sys/socket.h>
    #include <arpa/inet.h>
    
    #define __USE_LARGEFILE64
    #include <sys/resource.h>
    
    static const char request[] =
        "GET / HTTP/1.1\r\n"
        "Accept: */*\r\n"
        "Accept-Encoding: gzip, deflate, br\r\n"
        "Accept-Language: en-US,en;q=0.5\r\n"
        "Cache-Control: no-cache\r\n"
        "Connection: keep-alive, Upgrade\r\n"
        "Pragma: no-cache\r\n"
        "Sec-Fetch-Dest: websocket\r\n"
        "Sec-Fetch-Mode: websocket\r\n"
        "Sec-Fetch-Site: cross-site\r\n"
        "Sec-WebSocket-Extensions: permessage-deflate\r\n"
        "Sec-WebSocket-Key: ui27BbCABJn5wWU2Jc1Ejw==\r\n"
        "Sec-WebSocket-Version: 13\r\n"
        "Upgrade: websocket\r\n"
        "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:104.0) Gecko/20100101 Firefox/104.0\r\n"
        "\r\n"
    ;
    
    int main(void)
    {
        struct rlimit l = {
            .rlim_cur = 8192,
            .rlim_max = 8192,
        };
        setrlimit(RLIMIT_NOFILE, &l);
    
        for (int i = 0;i < 5000;i++) {
            int fd = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK | SOCK_CLOEXEC, 0);
    
            struct sockaddr_in addr = {
                .sin_addr.s_addr = inet_addr("127.0.0.1"),
                .sin_family = AF_INET,
                .sin_port = htons(8080),
            };
    
            connect(fd, (struct sockaddr *)&addr , sizeof(addr));
            send(fd, request, sizeof(request) - 1, 0);
        }
    
        sleep(3);
        abort();
    
        return 0;
    }
    

    Server output will be something like this:

    lwan/build$ ./src/samples/hello/hello
    2502216 lwan-job.c:101 lwan_job_thread_init() Initializing low priority job thread
    2502216 lwan-tables.c:46 lwan_tables_init() Uncompressing MIME type table: 8489->31379 bytes, 957 entries
    2502216 lwan.c:594 setup_from_config() Loading configuration file: hello.conf
    2502216 lwan-config.c:803 config_open_path() Could not open configuration file: hello.conf: No such file or directory (error number 2)
    2502216 lwan-response.c:77 lwan_response_init() Initializing default response
    2502216 lwan.c:842 lwan_init_with_config() Initializing lwan web server
    2502216 lwan.c:868 lwan_init_with_config() Using 16 threads, maximum 65536 sockets per thread
    2502216 lwan-readahead.c:186 lwan_readahead_init() Initializing low priority readahead thread
    2502216 lwan-thread.c:1257 lwan_thread_init() Initializing threads
    2502216 lwan-thread.c:1275 lwan_thread_init() 16 CPUs of 16 are online. Reading topology to pre-schedule clients
    2502218 lwan-thread.c:901 thread_io_loop() Worker thread #1 starting
    2502216 lwan-socket.c:206 listen_addrinfo() Listening on http://127.0.0.1:8080
    2502219 lwan-thread.c:901 thread_io_loop() Worker thread #2 starting
    2502220 lwan-thread.c:901 thread_io_loop() Worker thread #3 starting
    2502221 lwan-thread.c:901 thread_io_loop() Worker thread #4 starting
    2502222 lwan-thread.c:901 thread_io_loop() Worker thread #5 starting
    2502223 lwan-thread.c:901 thread_io_loop() Worker thread #6 starting
    2502224 lwan-thread.c:901 thread_io_loop() Worker thread #7 starting
    2502225 lwan-thread.c:901 thread_io_loop() Worker thread #8 starting
    2502226 lwan-thread.c:901 thread_io_loop() Worker thread #9 starting
    2502227 lwan-thread.c:901 thread_io_loop() Worker thread #10 starting
    2502228 lwan-thread.c:901 thread_io_loop() Worker thread #11 starting
    2502229 lwan-thread.c:901 thread_io_loop() Worker thread #12 starting
    2502230 lwan-thread.c:901 thread_io_loop() Worker thread #13 starting
    2502231 lwan-thread.c:901 thread_io_loop() Worker thread #14 starting
    2502232 lwan-thread.c:901 thread_io_loop() Worker thread #15 starting
    2502233 lwan-thread.c:901 thread_io_loop() Worker thread #16 starting
    2502216 lwan-thread.c:1360 lwan_thread_init() Worker threads created and ready to serve
    2502216 lwan.c:909 lwan_main_loop() Ready to serve
    2502224 lwan-thread.c:943 thread_io_loop() HUP fd: 1414 flags: 64, 1
    2502227 lwan-thread.c:943 thread_io_loop() HUP fd: 1401 flags: 64, 1
    ...
    2512259 lwan-thread.c:943 thread_io_loop() HUP fd: 87 flags: 46, 1
    2512259 lwan-thread.c:943 thread_io_loop() HUP fd: 71 flags: 46, 1
    2512259 lwan-thread.c:943 thread_io_loop() HUP fd: 55 flags: 46, 1
    

    This means connections were closed by test application abort well. Until this everything went fine.

    After keep-alive timeout reached Lwan starts sending these messages:

    2512261 lwan-tq.c:100 timeout_queue_expire() close error, fd: 889
    2512261 lwan-tq.c:100 timeout_queue_expire() close error, fd: 73
    2512261 lwan-tq.c:100 timeout_queue_expire() close error, fd: 41
    ...
    

    Unfortunately this means sometimes that close() in the timeout_queue_expire() function hit another valid connection was created by another http request.

    After I reverted back this commit (https://github.com/lpereira/lwan/commit/7a8a0eb7023122ad9605627c45af03b78fca44c4) everything get back to normal.

    Thanks, pontscho

    opened by pontscho 3
  • Debian stable build fails not finding `lua.h`

    Debian stable build fails not finding `lua.h`

    I hope I have followed the build instructions correctly. I cannot get lwan to build; it is looking for a lua.h, but there must be an option -I/usr/include/lua5.1 missing somewhere. Cmake finds that Lua 5.1.3 is installed, but make cannot build.

    Running on Debian stable (bullseye).

    Attached is the full output from cmake and then make in a pristine build directory.

    nr@homedog ~/n/l/build> cmake ..
    -- The C compiler identification is GNU 10.2.1
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working C compiler: /usr/bin/cc - skipped
    -- Detecting C compile features
    -- Detecting C compile features - done
    -- Running CMake for lwan (Scalable, high performance, experimental web server)
    -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2") 
    -- No build type selected, defaulting to Debug
    -- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11") 
    -- Looking for pthread.h
    -- Looking for pthread.h - found
    -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
    -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
    -- Looking for pthread_create in pthreads
    -- Looking for pthread_create in pthreads - not found
    -- Looking for pthread_create in pthread
    -- Looking for pthread_create in pthread - found
    -- Found Threads: TRUE  
    -- Found Python3: /usr/include/python3.9 (found suitable version "3.9.2", minimum required is "3.9") found components: Development Development.Module Development.Embed 
    -- Building with WSGI support using 
    -- Checking for modules 'luajit>=2.0;luajit<2.2'
    --   No package 'luajit' found
    --   No package 'luajit' found
    -- Checking for modules 'lua>=5.1.0;lua<=5.1.999'
    --   Found lua, version 5.1.3
    --   Found lua, version 5.1.3
    -- Building with Lua support using lua;m;dl
    -- Checking for modules 'libbrotlienc;libbrotlidec;libbrotlicommon'
    --   Found libbrotlienc, version 1.0.9
    --   Found libbrotlidec, version 1.0.9
    --   Found libbrotlicommon, version 1.0.9
    -- Checking for module 'libzstd'
    --   Found libzstd, version 1.4.8
    -- Looking for linux/tls.h
    -- Looking for linux/tls.h - found
    -- mbedTLS not found: not building with TLS support
    -- Looking for linux/capability.h
    -- Looking for linux/capability.h - found
    -- Looking for sys/auxv.h
    -- Looking for sys/auxv.h - found
    -- Looking for sys/epoll.h
    -- Looking for sys/epoll.h - found
    -- Looking for 3 include files sys/time.h, ..., sys/event.h
    -- Looking for 3 include files sys/time.h, ..., sys/event.h - not found
    -- Looking for alloca.h
    -- Looking for alloca.h - found
    -- Looking for getauxval
    -- Looking for getauxval - found
    -- Looking for get_current_dir_name
    -- Looking for get_current_dir_name - found
    -- Looking for reallocarray
    -- Looking for reallocarray - found
    -- Looking for mempcpy
    -- Looking for mempcpy - found
    -- Looking for memrchr
    -- Looking for memrchr - found
    -- Looking for pipe2
    -- Looking for pipe2 - found
    -- Looking for accept4
    -- Looking for accept4 - found
    -- Looking for readahead
    -- Looking for readahead - found
    -- Looking for mkostemp
    -- Looking for mkostemp - found
    -- Looking for clock_gettime
    -- Looking for clock_gettime - found
    -- Looking for pthread_barrier_init
    -- Looking for pthread_barrier_init - found
    -- Looking for pthread_set_name_np
    -- Looking for pthread_set_name_np - not found
    -- Looking for posix_fadvise
    -- Looking for posix_fadvise - found
    -- Looking for getentropy
    -- Looking for getentropy - found
    -- Looking for fwrite_unlocked
    -- Looking for fwrite_unlocked - found
    -- Looking for gettid
    -- Looking for gettid - found
    -- Looking for secure_getenv
    -- Looking for secure_getenv - found
    -- Looking for statfs
    -- Looking for statfs - found
    -- Looking for syslog
    -- Looking for syslog - found
    -- Looking for dladdr
    -- Looking for dladdr - not found
    -- Performing Test LWAN_HAVE_STD_GNU99
    -- Performing Test LWAN_HAVE_STD_GNU99 - Success
    -- Performing Test LWAN_HAVE_BUILTIN_CPU_INIT
    -- Performing Test LWAN_HAVE_BUILTIN_CPU_INIT - Success
    -- Performing Test LWAN_HAVE_BUILTIN_EXPECT_PROBABILITY
    -- Performing Test LWAN_HAVE_BUILTIN_EXPECT_PROBABILITY - Success
    -- Performing Test LWAN_HAVE_BUILTIN_CLZLL
    -- Performing Test LWAN_HAVE_BUILTIN_CLZLL - Success
    -- Performing Test LWAN_HAVE_BUILTIN_FPCLASSIFY
    -- Performing Test LWAN_HAVE_BUILTIN_FPCLASSIFY - Success
    -- Performing Test LWAN_HAVE_BUILTIN_MUL_OVERFLOW
    -- Performing Test LWAN_HAVE_BUILTIN_MUL_OVERFLOW - Success
    -- Performing Test LWAN_HAVE_BUILTIN_ADD_OVERFLOW
    -- Performing Test LWAN_HAVE_BUILTIN_ADD_OVERFLOW - Success
    -- Performing Test LWAN_HAVE_STATIC_ASSERT
    -- Performing Test LWAN_HAVE_STATIC_ASSERT - Success
    -- Performing Test LWAN_HAVE_SO_ATTACH_REUSEPORT_CBPF
    -- Performing Test LWAN_HAVE_SO_ATTACH_REUSEPORT_CBPF - Success
    -- Performing Test LWAN_HAVE_SO_INCOMING_CPU
    -- Performing Test LWAN_HAVE_SO_INCOMING_CPU - Success
    -- Building with Valgrind support
    -- Performing Test LWAN_HAVE_MTUNE_NATIVE
    -- Performing Test LWAN_HAVE_MTUNE_NATIVE - Success
    -- Performing Test LWAN_HAVE_MARCH_NATIVE
    -- Performing Test LWAN_HAVE_MARCH_NATIVE - Success
    -- Performing Test LWAN_HAVE_STACK_PROTECTOR_EXPLICIT
    -- Performing Test LWAN_HAVE_STACK_PROTECTOR_EXPLICIT - Success
    -- Performing Test LWAN_HAVE_IMMEDIATE_BINDING
    -- Performing Test LWAN_HAVE_IMMEDIATE_BINDING - Success
    -- Performing Test LWAN_HAVE_READ_ONLY_GOT
    -- Performing Test LWAN_HAVE_READ_ONLY_GOT - Success
    -- Performing Test LWAN_HAVE_NO_PLT
    -- Performing Test LWAN_HAVE_NO_PLT - Success
    -- Performing Test LWAN_HAVE_NO_PIE
    -- Performing Test LWAN_HAVE_NO_PIE - Success
    -- Performing Test LWAN_HAVE_NOEXEC_STACK
    -- Performing Test LWAN_HAVE_NOEXEC_STACK - Success
    -- Building without a sanitizer
    -- Performing Test supports -Wduplicated-cond
    -- Performing Test supports -Wduplicated-cond - Success
    -- Performing Test supports -Wduplicated-branches
    -- Performing Test supports -Wduplicated-branches - Success
    -- Performing Test supports -Wlogical-op
    -- Performing Test supports -Wlogical-op - Success
    -- Performing Test supports -Wrestrict
    -- Performing Test supports -Wrestrict - Success
    -- Performing Test supports -Wdouble-promotion
    -- Performing Test supports -Wdouble-promotion - Success
    -- Performing Test supports -Wno-unused-parameter
    -- Performing Test supports -Wno-unused-parameter - Success
    -- Performing Test supports -Wstringop-truncation
    -- Performing Test supports -Wstringop-truncation - Success
    -- Performing Test supports -Wvla
    -- Performing Test supports -Wvla - Success
    -- Performing Test supports -Wunsequenced
    -- Performing Test supports -Wunsequenced - Failed
    -- Performing Test supports -Wno-free-nonheap-object
    -- Performing Test supports -Wno-free-nonheap-object - Success
    -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.9.2", minimum required is "3") 
    -- Using built-in context switching routines for x86_64 processors
    -- Using Brotli for mimegen
    -- Checking for module 'sqlite3>=3.6.20'
    --   Found sqlite3, version 3.34.1
    -- Not building benchmark suite: database libraries not found.
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /home/nr/net/lwan/build
    nr@homedog ~/n/l/build> make
    Scanning dependencies of target bin2hex
    [  1%] Building C object src/bin/tools/CMakeFiles/bin2hex.dir/bin2hex.c.o
    [  2%] Linking C executable bin2hex
    [  2%] Built target bin2hex
    Scanning dependencies of target generate_auto_index_icons
    [  3%] Bundling auto-index icons
    [  3%] Built target generate_auto_index_icons
    Scanning dependencies of target mimegen
    [  4%] Building C object src/bin/tools/CMakeFiles/mimegen.dir/mimegen.c.o
    [  5%] Building C object src/bin/tools/CMakeFiles/mimegen.dir/__/__/lib/hash.c.o
    In file included from /home/nr/net/lwan/src/lib/hash.c:35:
    /home/nr/net/lwan/src/lib/lwan-private.h:182:10: fatal error: lua.h: No such file or directory
      182 | #include <lua.h>
          |          ^~~~~~~
    compilation terminated.
    make[2]: *** [src/bin/tools/CMakeFiles/mimegen.dir/build.make:95: src/bin/tools/CMakeFiles/mimegen.dir/__/__/lib/hash.c.o] Error 1
    make[1]: *** [CMakeFiles/Makefile2:654: src/bin/tools/CMakeFiles/mimegen.dir/all] Error 2
    make: *** [Makefile:149: all] Error 2
    
    opened by nrnrnr 2
  • WebSocket connection error with Apple Safari

    WebSocket connection error with Apple Safari

    Hi,

    first of all thanks for clarifying my previous patch about WebSocket ping.

    I found and another not-too-fun "issue" with Lwan's web socket implementation, which isn't a real bug in Lwan but in Apple Safari. For some unknown reasons it isn't send Upgrade header when it starts a new connection but if I comment out lines from lwan-request.c:1344-1347:

         const char *upgrade = lwan_request_get_header(request, "Upgrade");
         if (UNLIKELY(!upgrade || !streq(upgrade, "websocket")))
             return HTTP_BAD_REQUEST;
    

    helps establish web socket connection with Lwan without any other problem.

    Commenting out these lines is just a workaround, not a real solution but helps to interoperatibihity between lwan and Safari and strict check isn’t get injured because the next few lines are used to check the websocket connection. If the sec-websocket-key field is missing, it either means a total error on any ways.

    What is your opinion about this case ?

    pontscho

    opened by pontscho 6
  • max_post_data_size

    max_post_data_size

    hello, I encounter some issue for post file.

    I am setting max_post_data_size = 40960000, and try to POST data as application/octet-stream.

    While I post a small content ~ 20kb, it work fine.

    However, when I try post around 2MB, I get 500 internal server error: The server encountered an internal error that couldn't be recovered from.

    What is the max size for max_post_data_size ?

    Not sure If I missing something ?

    opened by kiwionly 1
  • Centos编译报错

    Centos编译报错

    lwan-request.c:(.text+0xe79): undefined reference to __builtin_expect_with_probability' lwan-request.c:(.text+0x12ad): undefined reference to__builtin_expect_with_probability' lwan-request.c:(.text+0x1485): undefined reference to __builtin_expect_with_probability' lwan-request.c:(.text+0x19b8): undefined reference to__builtin_expect_with_probability' lwan-request.c:(.text+0x1b7a): undefined reference to __builtin_expect_with_probability' ../../lib/liblwan.a(lwan-request.c.o):lwan-request.c:(.text+0x1d79): more undefined references to__builtin_expect_with_probability' follow collect2: error: ld returned 1 exit status make[2]: *** [src/bin/lwan/lwan] Error 1 make[1]: *** [src/bin/lwan/CMakeFiles/lwan.dir/all] Error 2 make: *** [all] Error 2

    opened by fishfly123 3
Releases(v0.5)
  • v0.5(May 28, 2022)

  • v0.4(Oct 1, 2021)

  • v0.3(Jan 28, 2020)

    Since last release: in the src directory alone, there were 83 files changed, 6225 lines added, 4343 lines removed.

    Code has gotten clearer, safer (thanks @ossfuzz!), and more efficient all around. Of note: a lot of memory allocations in the fast path are gone; deferred callbacks for coroutines are really cheap now; parsers (HTTP, configuration, template) are much more robust, coroutine context switch is pretty decent (~22ns on a SNB processor), zstd+brotli support, event loop has been completely overhauled, a few tweaks in the template engine, etc; the list goes on.

    A lot of changes were covered in details in this blog post.

    Source code(tar.gz)
    Source code(zip)
  • v0.2(Jan 18, 2019)

  • v0.1(Dec 10, 2017)

Owner
Leandro A. F. Pereira
Leandro A. F. Pereira
Embeddable Event-based Asynchronous Message/HTTP Server library for C/C++

libasyncd Embeddable Event-based Asynchronous Message/HTTP Server library for C/C++. What is libasyncd? Libasyncd is an embeddable event-driven asynch

Seungyoung 166 Dec 5, 2022
A http/websocket server framework on linux.

The framework is a Web-Server on unix based system. Without using any third-party libraries, the framework writes from unix system calls and standard C library functions.

xingyuuchen 17 Oct 15, 2022
🌱Light and powerful C++ web framework for highly scalable and resource-efficient web application. It's zero-dependency and easy-portable.

Oat++ News Hey, meet the new oatpp version 1.2.5! See the changelog for details. Check out the new oatpp ORM - read more here. Oat++ is a modern Web F

Oat++ 6k Jan 4, 2023
Your high performance web application C framework

facil.io is a C micro-framework for web applications. facil.io includes: A fast HTTP/1.1 and Websocket static file + application server. Support for c

Bo 1.7k Dec 29, 2022
A high-performance REST Toolkit written in C++

Pistache Pistache is a modern and elegant HTTP and REST framework for C++. It is entirely written in pure-C++14 and provides a clear and pleasant API.

null 2.8k Dec 29, 2022
Cetus is a high performance, stable, protocol aware proxy for MySQL Group Replication.

Introduction Cetus is a high performance, stable, protocol aware proxy for MySQL Group Replication. Getting started 1. Prerequisites cmake gcc glib2-d

null 37 Sep 19, 2022
CppCMS - High Performance C++ Web Framework

CppCMS - High Performance C++ Web Framework What is CppCMS? CppCMS is a Free High Performance Web Development Framework (not a CMS) aimed at Rapid Web

Artyom Beilis 375 Dec 25, 2022
Drogon: A C++14/17 based HTTP web application framework running on Linux/macOS/Unix/Windows

English | 简体中文 | 繁體中文 Overview Drogon is a C++14/17-based HTTP application framework. Drogon can be used to easily build various types of web applicat

An Tao 8.5k Dec 31, 2022
C library to create simple HTTP servers and Web Applications.

Onion http server library Travis status Coverity status Onion is a C library to create simple HTTP servers and Web Applications. master the developmen

David Moreno Montero 1.9k Dec 31, 2022
bittyhttp - A threaded HTTP library for building REST services in C.

bittyhttp - A threaded HTTP library for building REST services in C.

Colin Luoma 12 Nov 29, 2021
Pistache is a modern and elegant HTTP and REST framework for C++

Pistache is a modern and elegant HTTP and REST framework for C++. It is entirely written in pure-C++17* and provides a clear and pleasant API.

null 2.8k Jan 4, 2023
TreeFrog Framework : High-speed C++ MVC Framework for Web Application

Small but Powerful and Efficient TreeFrog Framework is a high-speed and full-stack web application framework based on C++ and Qt, which supports HTTP

TreeFrog Framework 1.1k Dec 22, 2022
Embedded C/C++ web server

CivetWeb The official home of CivetWeb is https://github.com/civetweb/civetweb Continuous integration for Linux and macOS (Travis CI): Continuous inte

null 2.3k Jan 8, 2023
The C++ REST SDK is a Microsoft project for cloud-based client-server communication in native code using a modern asynchronous C++ API design. This project aims to help C++ developers connect to and interact with services.

Welcome! The C++ REST SDK is a Microsoft project for cloud-based client-server communication in native code using a modern asynchronous C++ API design

Microsoft 7.2k Jan 8, 2023
A C++11 RESTful web server library

Served Overview Served is a C++ library for building high performance RESTful web servers. Served builds upon Boost.ASIO to provide a simple API for d

Meltwater 696 Dec 28, 2022
A lightweight Bedorck Dedicated Server Plugin Loader

LiteLoader 简体中文 A lightweight Bedorck Dedicated Server Plugin Loader Based on BedrockX Install Download LiteLoader from Releases or Actions, unzip it

null 572 Dec 31, 2022
cserv is an event-driven and non-blocking web server

cserv is an event-driven and non-blocking web server. It ideally has one worker process per cpu or processor core, and each one is capable of handling thousands of incoming network connections per worker. There is no need to create new threads or processes for each connection.

null 43 Nov 6, 2022