I had to make the following changes to get libphenom to build on ubuntu 12.04. These changes seem reasonable to me, but let me know if there's a better way to do this.

Hi, there is a compilation error:

OS X 10.8.5 Installed Autotools (autoconf, automake, libtool) Installed c-ares (with macport)

$ pkg-config --libs libcares
-L/opt/local/lib -lcares

$ ./autogen.sh
$ ./configure
$ make

$ make check
make  check-TESTS
PASS: tests/counter.t
PASS: tests/memory.t
PASS: tests/timer.t
PASS: tests/printf.t
PASS: tests/iobasic.t
PASS: tests/stream.t
PASS: tests/tpool.t
PASS: tests/string.t
PASS: tests/hashtable.t
PASS: tests/sockaddr.t
./test-driver: line 95: 16030 Segmentation fault: 11  "$@" > $log_file 2>&1
FAIL: tests/dns.t
PASS: tests/variant.t
PASS: tests/bench/iopipes.t
============================================================================
Testsuite summary for phenom 0.1
============================================================================
# TOTAL: 13
# PASS:  12
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log
============================================================================
make[2]: *** [test-suite.log] Error 1
make[1]: *** [check-TESTS] Error 2
make: *** [check-am] Error 2

The content of the test-suite.log is:

==================================
   phenom 0.1: ./test-suite.log
==================================

# TOTAL: 13
# PASS:  12
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: tests/dns.t
=================

#     Failed test (tests/dns.c:check_addrinfo_result() at line 46)

Please help~

This allow us to preserve more state about the listener object when we handle an accept.

For example, I have a module that handles http requests a bit like sinatra. I'd like to have multiple instances of this module, so that a request to / on two different ports can be handled two different ways. Without being able to store data in the listener object, there's no nice way to figure out how we should handle a given request.

when i set 67108864 or more elements, the hashtalbe crash.

(gdb) p *ht $5 = {nelems = 67108864, table_size = 134217728, elem_size = 16, mask = 134217727, kdef = 0x7ffff7ff6ea0, vdef = 0x7ffff7ff68e0, table = 0x7ffe68e92020 ""}

my init: reph = ph_ht_init(&(mth->ht), 10000, &ph_ht_string_key_def, &ph_ht_uint32_val_def);

Hey guys,

Libphenom is currently refusing to compile on OS X 10.10, both with Clang and (GNU) gcc.

Here's a gist of the build log.

Using:

OpenSSL 1.0.1j Concurrencykit 0.4.4 Automake 1.14.1 Autoconf 2.69 GNU Libtool 2.4.2 Pkg-config 0.28

If we want a POSIX 2008 environment as requested by _POSIX_C_SOURCE here, _XOPEN_SOURCE should be 700 so they are in agreement.

See: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap02.html#tag_02_01_04

Fixes build on FreeBSD 10 which redefines _POSIX_C_SOURCE depending on _XOPEN_SOURCE def.

First commit contains the changes necessary for ck 0.3.3. Would appreciate some peer review here!

• In upstream, ck_ring changed quite a bit. ck_ring_init() doesn't touch the buffer, and it was removed from ck_ring_t. Instead, a new opaque ck_ring_buffer_t must be separately initialized and is passed in to each enqueue() and dequeue(). https://github.com/sbahra/ck/commit/3edb523da5f6c9dcea41879ceb6643dcc7bde305

The rest is a switch to using git subtree to vendorize ck. For a good overview, http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/. IMO, this is the least smelly way to vendor something, carrying the full repo is fine, and this gives the full power of git for maintenance of the vendor branch.

Why ph_string_concatenate function is not included in the libPhenom??

ph_string_t *ph_string_concatenate(ph_string_t *a,ph_string_t *b) { uint32_t len1=strlen(a); uint32_t len2=strlen(b); uint32_t i;

  for(i=len1;i<len1+len2;i++)
 {
          a->buf[i]=b->buf[i];
 }

 return(a);

}

Not able to reproduce this on every run, but here's a backtrace:

Program exited normally.
(gdb) run
Starting program: /Users/jbergstroem/Work/libphenom/tests/iobasic.t
1..13
ok 1 - PH_OK == ph_nbio_init(0)
ok 2 - PH_OK == ph_job_init(&pipe_job)
ok 3 - 0 == ph_pipe(pipe_fd, PH_PIPE_NONBLOCK)
ok 4 - set up ping
ok 5 - 100ms resolution: diff=100
ok 6 - 1 == read(pipe_fd[0], &buf, sizeof(buf))
ok 7 - 100ms resolution: diff=101

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000004
[Switching to process 21613 thread 0x10b]
0x00000001000165f5 in ck_epoch_synchronize (global=<value temporarily unavailable, due to optimizations>, record=0x0) at thirdparty/ck/src/ck_epoch.c:395
395     record->epoch = delta;
(gdb) bt
#0  0x00000001000165f5 in ck_epoch_synchronize (global=<value temporarily unavailable, due to optimizations>, record=0x0) at thirdparty/ck/src/ck_epoch.c:395
#1  0x000000010001660e in ck_epoch_barrier (global=0x100020600, record=0x0) at thirdparty/ck/src/ck_epoch.c:403
#2  0x000000010000d149 in destroy_thread (ptr=0x101200008) at corelib/thread.c:108
#3  0x00000001001104b0 in _pthread_tsd_cleanup ()
#4  0x00000001001099d2 in _pthread_exit ()
#5  0x000000010010977d in _pthread_start ()
#6  0x00000001000f61a1 in thread_start ()
Current language:  auto; currently minimal

This removes the thirdparty/ck directory and uses pkg-config to specify the concurrency-kit dependency. It forces ck to be <= 0.3.2, as that's the last compatible release with the current libphenom.

For more reference, see #46 and the comments in #59.

I have a libphenom application that uses concurrency-kit extensively. It needs some things from ck that are not available in 0.3.0. This upgrades to ck-0.4.1 to make those things available.

There were some changes to libphenom that were necessary due to breaking api changes in ck (https://github.com/sbahra/ck/commit/3edb523da5f6c9dcea41879ceb6643dcc7bde305). I've updated job.c and job.h to use the new api.

The libphenom tests pass, and I haven't seen any issues in my application level testing.

Some questions:

1. Does libphenom have a better way to update ck than copying in the code for the new tag?
2. Is there a preferred way to benchmark these changes to make sure they're not regressions?

I got the following compile error when doing ./autogen.sh; ./configure; make. I did a git clone on this repo and the github repo on concurrencyKit.

Any ideas? Thanks.

libtool: compile:  gcc -DHAVE_CONFIG_H -Ithirdparty/ck/include -Iinclude -I. -DPHENOM_IMPL=1 -Werror -m64 -fno-omit-frame-pointer -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D__x86_64__ -I/usr/local/include -std=gnu1x -pedantic -Wall -Wextra -Wno-declaration-after-statement -W -Wundef -Wendif-labels -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wno-nested-externs -Winline -Wdisabled-optimization -fstrict-aliasing -pipe -Wno-parentheses -O2 -g -gdwarf-2 -MT corelib/libphenom_la-counter.lo -MD -MP -MF corelib/.deps/libphenom_la-counter.Tpo -c corelib/counter.c  -fPIC -DPIC -o corelib/.libs/libphenom_la-counter.o
In file included from include/phenom/counter.h:81:0,
                 from corelib/counter.c:17:
include/phenom/defs.h:368:41: error: static assertion failed: "counter_iterator_definition_bad"
 #  define ph_static_assert(expr, msg)   _Static_assert(expr, #msg)
                                         ^
corelib/counter.c:170:1: note: in expansion of macro 'ph_static_assert'
 ph_static_assert(sizeof(struct ph_counter_scope_iterator)

Hello,

It’d be very helpful if you could tag a release so we know the code there is stable and can be used. I work on Homebrew and we provide a way to install libphenom but we rely on the git HEAD so can’t provide checksums and can’t ensure users will always get the same code depending on the time they install it, making it harder for us to debug issues.

Thanks!

I haven't dug into this yet. I'm not getting these consistently, sometime the run will be successful. The affinity failed message is always present ("nbio" and "job" in the message were added by me).

1387793688.893 err: /1 nbio failed to set thread 0x80240e200 affinity
1387793688.893 err: /2 nbio failed to set thread 0x80280e200 affinity
1387793688.893 err: /3 nbio failed to set thread 0x802c0e200 affinity
1387793688.894 err: /4 nbio failed to set thread 0x80300e200 affinity
1387793688.894 err: /5 nbio failed to set thread 0x80340e200 affinity
1387793688.894 err: /6 nbio failed to set thread 0x80380e200 affinity
1387793688.894 err: /7 nbio failed to set thread 0x803c0e200 affinity
1387793688.894 err: /8 nbio failed to set thread 0x80400e200 affinity
1387793688.894 err: /9 nbio failed to set thread 0x80440e200 affinity
1387793688.894 err: /10 nbio failed to set thread 0x80480e200 affinity
1387793688.894 err: /11 nbio failed to set thread 0x804c0e200 affinity
1387793688.895 err: dns/13 job failed to set thread 0x80540e200 affinity
1387793688.895 err: dns/12 job failed to set thread 0x80500e200 affinity
1387793688.893 err: phenom:sched/0 nbio failed to set thread 0x80209a200 affinity
#     Failed test (tests/timer.c:record_tick() at line 38)
#     Failed test (tests/timer.c:record_tick() at line 38)
# Looks like you failed 2 tests of 8.
# ELAPSED: 749ms

1..8
ok 1 - PH_OK == ph_nbio_init(0)
ok 2 - PH_OK == ph_job_init(&timer)
ok 3 - PH_OK == ph_job_set_timer_at(&timer, last_tick)
ok 4 - 100ms resolution: diff=106
ok 5 - 100ms resolution: diff=106
not ok 6 - 100ms resolution: diff=213
not ok 7 - 100ms resolution: diff=214
ok 8 - PH_OK == ph_sched_run()

Throwing this one out there for folks that want to dive in.

In the bufq API we have ph_bufq_consume_record which searches the bufq for a matching record delimiter.

For many internet protocols this delimiter is CRLF. For many other applications, we're likely to be looking for LF.

There are a couple of optimizations that could be investigated.

• Can we accelerate the memmem call using the sse3_memchr function from here? http://repnop.org/carte/snippets/simd/. The performance.data file indicates that this performs consistently better than the darwin libc. To adopt this, we'd need to detect sse3 either at runtime or compile time
• are there specializations of sse3_memchr that can be made for detecting CRLF?
• are there specializations of string matching algorithms with a constant, known needle that we could use?
• is there a more efficient way to match needles across the "straddle" point in cases where the delimiter straddles discontiguous memory regions?
• for long records, we make repeated calls and search across the same memory regions repeatedly. We can surely cache the last searched offset and improve efficiency. We'd need to make sure that we invalidate this offset in the appropriate circumstances (mostly when we've consumed past that point)