WANProxy trunk may be unstable: basic multithreading support merged.

Juli Mallett juli at clockworksquid.com
Thu Jun 27 22:22:51 PDT 2013


All,

The multithreading branch (^/branches/mt) has been merged into trunk.
 This splits some tasks, like I/O, polling and timeout handling, into
their own threads.  For some workloads this is probably a
pessimization, for others it is a clear improvement.  (For artificial
benchmarks and in particular microbenchmarks the costs of
synchronization are certainly apparent, but hopefully not too
onerous.)

Note that unfortunately right now only FreeBSD and Mac OS X and other
systems using kqueue are supported by trunk, as I have not added some
infrastructure that is required to support polling methods other than
kqueue with multithreading.)

I'd encourage you to test it soon if you have any intention of running
trunk, so that I can address any issues that come up while I'm still
working on it.

These changes are essential for work that is due to be committed in
the near future which will make it possible to use WANProxy as a
transparent proxy (and the WANProxy codebase as a basis for other
kinds of transparent proxies), which by necessity may do some work in
other threads, and so needed the whole event system to be robust in
the face of multithreading.

While the immediate benefits other than enabling eventual support for
transparent proxying may seem a little unclear, I would note that this
also will make it easier to do things like XCodec processing in other
threads, as well as running cryptographic algorithms in their own
threads, which will be more important in the future.  We may also be
able to generally distribute all tasks running within WANProxy across
multiple threads, which would be very beneficial in scaling to larger
installations.

My apologies for breaking trunk on everything that doesn't use kqueue,
and for any regressions that I may have introduced.  I know that at a
minimum things don't seem to be shutting down very gracefully on exit,
and while that's a difficult problem and one I need to put more time
into, it isn't as urgent as some other things, yet.

Thanks,
Juli.



More information about the wanproxy mailing list