[wanproxy/wanproxy] c5282d: Splice out cancellation from abstract Actions.

Juli Mallett juli at clockworksquid.com
Wed Jan 27 19:02:30 PST 2016


  Branch: refs/heads/master
  Home:   https://github.com/wanproxy/wanproxy
  Commit: c5282dde483c1fb77cb9783041ebf3ab5d6debe3
      https://github.com/wanproxy/wanproxy/commit/c5282dde483c1fb77cb9783041ebf3ab5d6debe3
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-26 (Tue, 26 Jan 2016)

  Changed paths:
    M event/action.h
    M event/callback_queue.h
    A event/cancellation.h
    M event/event_poll.h
    M io/pipe/pipe_producer.cc
    M io/pipe/pipe_splice.cc
    M io/pipe/splice.cc
    M io/pipe/splice_pair.cc
    M io/socket/socket_uinet.cc
    M programs/wanproxy/ssh_stream.cc
    M ssh/ssh_transport_pipe.cc

  Log Message:
  -----------
  Splice out cancellation from abstract Actions.


  Commit: de6d7ef7efb61e24091313e3631122f2f4c984af
      https://github.com/wanproxy/wanproxy/commit/de6d7ef7efb61e24091313e3631122f2f4c984af
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-26 (Tue, 26 Jan 2016)

  Changed paths:
    M event/example/callback-speed1/callback-speed1.cc
    M event/speed_test.h

  Log Message:
  -----------
  Fix SpeedTest locking.


  Commit: 30435f525efe2c0907d16ffb60dbb15d75fd30a5
      https://github.com/wanproxy/wanproxy/commit/30435f525efe2c0907d16ffb60dbb15d75fd30a5
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-26 (Tue, 26 Jan 2016)

  Changed paths:
    M event/action.h
    M event/callback_queue.h
    M event/cancellation.h
    M event/event_poll.h

  Log Message:
  -----------
  Action::cancel no longer implies delete.

This removes unneeded indirection, and moves delete this into each
implementation that cares (for now) about being deleted at time of
cancel.


  Commit: 5ed9e9e1c29b638923f77a5cc3e9f3829bc28d63
      https://github.com/wanproxy/wanproxy/commit/5ed9e9e1c29b638923f77a5cc3e9f3829bc28d63
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-26 (Tue, 26 Jan 2016)

  Changed paths:
    M event/callback.h
    M event/callback_thread.cc

  Log Message:
  -----------
  Add CallbackBase::cancel which cancels a callback.

This takes the place of explicitly deleting a spoiled callback.


  Commit: e2fb98fa33a5fbdc6738fdb4261d65dbec4a7102
      https://github.com/wanproxy/wanproxy/commit/e2fb98fa33a5fbdc6738fdb4261d65dbec4a7102
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-26 (Tue, 26 Jan 2016)

  Changed paths:
    M event/example/callback-manyspeed1/callback-manyspeed1.cc

  Log Message:
  -----------
  Juse use CallbackThread for callback-manyspeed1.


  Commit: 23962a5c03ff34a1e47dc9fa6861516baa9e0041
      https://github.com/wanproxy/wanproxy/commit/23962a5c03ff34a1e47dc9fa6861516baa9e0041
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/object_callback.h
    M event/typed_callback.h
    M event/typed_pair_callback.h
    M io/pipe/splice_pair.cc
    M io/pipe/splice_pair.h
    M programs/wanproxy/proxy_connector.cc
    M programs/wanproxy/proxy_connector.h
    M programs/wanproxy/ssh_proxy_connector.cc
    M programs/wanproxy/ssh_proxy_connector.h

  Log Message:
  -----------
  Eliminate argument-capture callbacks.

These served to carry state to shared callback methods, but in
practice these were not used widely, and where they did they
carried a kind of state that was really not optimal.  Instead,
indicate things like which socket has been closed by having a
different method for each.  These methods are easier to
understand and clearer in purpose.


  Commit: 3a4e42ff826738b9a64fd0dad92deb4f1922472b
      https://github.com/wanproxy/wanproxy/commit/3a4e42ff826738b9a64fd0dad92deb4f1922472b
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/cancellation.h
    M http/http_server_pipe.cc
    M http/http_server_pipe.h
    M io/io_system.cc
    M io/io_system.h
    M io/io_system_handle.cc
    M io/net/tcp_client.cc
    M io/net/tcp_client.h
    M io/net/udp_client.cc
    M io/net/udp_client.h
    M io/pipe/pipe_producer.cc
    M io/pipe/pipe_producer.h
    M io/pipe/pipe_splice.cc
    M io/pipe/pipe_splice.h
    M io/pipe/splice.cc
    M io/pipe/splice.h
    M io/pipe/splice_pair.cc
    M io/pipe/splice_pair.h
    M io/socket/socket_handle.cc
    M io/socket/socket_handle.h
    M programs/wanproxy/ssh_stream.cc
    M programs/wanproxy/ssh_stream.h
    M ssh/ssh_transport_pipe.cc
    M ssh/ssh_transport_pipe.h

  Log Message:
  -----------
  Eliminate some ephemeral cancellations.

This makes cancellations persist for cases where there is not a
variable parameter to be captured.  Only cancellations which capture
a parameter are now ephemeral.


  Commit: 176f2929f8d347e33dc53cfb1483cd6f4e1ae684
      https://github.com/wanproxy/wanproxy/commit/176f2929f8d347e33dc53cfb1483cd6f4e1ae684
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/cancellation.h

  Log Message:
  -----------
  Document cancellations and a to-do item.


  Commit: abc8b96fcc0874154ea9099b65364bf61b062eef
      https://github.com/wanproxy/wanproxy/commit/abc8b96fcc0874154ea9099b65364bf61b062eef
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/callback.cc
    M event/callback.h
    M event/callback_queue.h
    M event/callback_thread.cc
    M event/event_system.cc
    M event/timeout_queue.cc

  Log Message:
  -----------
  Use persistent cancellations for CallbackThread.

This reverses the cancellation so that callbacks track whether they
have been scheduled, and cancel themselves with the scheduler they
are attached to.

This also pins callbacks to a scheduler after their first schedule.

This results in something like a 10% reduction in overhead on
callbacks and scheduling.  Further improvements are expected once
callbacks themselves persist.


  Commit: 70ec27847436b49636b4ded1a0d1b1484e24bb89
      https://github.com/wanproxy/wanproxy/commit/70ec27847436b49636b4ded1a0d1b1484e24bb89
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/cancellation.h

  Log Message:
  -----------
  Remove ephemeral cancellations.


  Commit: 71f4bdb0c9f0e1e808793ad3aa8e1940b5740b82
      https://github.com/wanproxy/wanproxy/commit/71f4bdb0c9f0e1e808793ad3aa8e1940b5740b82
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    R event/condition.h
    R event/dropbox.h
    R event/event_condition.h
    M event/test/Makefile
    R event/test/dropbox1/Makefile
    R event/test/dropbox1/dropbox1.cc
    R event/test/event-condition1/Makefile
    R event/test/event-condition1/event-condition1.cc
    R event/typed_condition.h

  Log Message:
  -----------
  Remove half-baked condition/dropbox API.


  Commit: bc66570ac5746d532132c62465b750cd29b75db9
      https://github.com/wanproxy/wanproxy/commit/bc66570ac5746d532132c62465b750cd29b75db9
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/example/Makefile
    R event/example/callback-manyspeed1/Makefile
    R event/example/callback-manyspeed1/callback-manyspeed1.cc

  Log Message:
  -----------
  Remove callback-manyspeed1.

It was useful for testing multithreading in isolation, but not now.


  Commit: f46d768660041ecc93ac8e7477f7bfd7b68a99b0
      https://github.com/wanproxy/wanproxy/commit/f46d768660041ecc93ac8e7477f7bfd7b68a99b0
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M LAYOUT
    M Makefile
    R network/Makefile
    R network/TODO
    R network/example/Makefile
    R network/example/network-packet-receive-hexdump1/Makefile
    R network/example/network-packet-receive-hexdump1/network-packet-receive-hexdump1.cc
    R network/network_interface.cc
    R network/network_interface.h
    R network/network_interface_pcap.cc
    R network/network_interface_pcap.h

  Log Message:
  -----------
  Get rid of defunct pcap code.


  Commit: 0d2194f69fc489a8fdfc929483c8bceb12d5480f
      https://github.com/wanproxy/wanproxy/commit/0d2194f69fc489a8fdfc929483c8bceb12d5480f
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/cancellation.h

  Log Message:
  -----------
  Don't assert lock ownership in Cancellation constructor.

This made sense when we were spawning ephemeral cancellations, but
not now, as we tend to construct cancellations during object creation.


  Commit: 2a3d74687234185a6f428092ea2c49b0c6716d14
      https://github.com/wanproxy/wanproxy/commit/2a3d74687234185a6f428092ea2c49b0c6716d14
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/callback.cc
    M event/callback.h
    M event/callback_queue.h
    M event/event_callback.h
    M event/event_poll.cc
    R event/object_callback.h
    M event/speed_test.h
    M http/http_server.cc
    M http/http_server.h
    M http/http_server_pipe.cc
    M io/net/tcp_client.cc
    M io/net/tcp_client.h
    M io/net/udp_client.cc
    M io/net/udp_client.h
    M io/socket/simple_server.h
    M io/socket/socket.h
    M programs/wanproxy/proxy_connector.cc
    M programs/wanproxy/proxy_connector.h
    M programs/wanproxy/proxy_socks_connection.cc
    M programs/wanproxy/proxy_socks_connection.h
    M programs/wanproxy/ssh_proxy_connector.cc
    M programs/wanproxy/ssh_proxy_connector.h
    M programs/wanproxy/ssh_stream.cc
    M programs/wanproxy/ssh_stream.h
    M ssh/ssh_transport_pipe.cc

  Log Message:
  -----------
  Checkpoint persistent callbacks.

This removes all central deletion of all callbacks, meaning that
any unconverted callbacks will leak memory.

This converts every SimpleCallback used in WANProxy into a
persistent callback associated with the object which performs the
callback.


  Commit: 7d919946cf2b1b2875dbec5197db90fab338175f
      https://github.com/wanproxy/wanproxy/commit/7d919946cf2b1b2875dbec5197db90fab338175f
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/callback.h
    M event/speed_test.h
    M http/http_server.h
    M io/net/tcp_client.h
    M io/net/udp_client.h
    M io/socket/simple_server.h
    M programs/wanproxy/proxy_connector.h
    M programs/wanproxy/proxy_socks_connection.h
    M programs/wanproxy/ssh_proxy_connector.h
    M programs/wanproxy/ssh_stream.h

  Log Message:
  -----------
  Nest the Method instance in SimpleCallback.

This is not really necessary for SimpleCallback, but is for consistency
with what the other callback types will go through.  This is to work
around the lack of typedef templates (and trying to avoid C++11-style
using declarations for now) for implementing things like EventCallback,
without exposing implementation details to consumers.


  Commit: 40c823d5943e96230a297d1b1e0e859d260df3a1
      https://github.com/wanproxy/wanproxy/commit/40c823d5943e96230a297d1b1e0e859d260df3a1
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M io/socket/test/Makefile
    M io/socket/test/socket-accept-connect1/Makefile
    M io/socket/test/socket-accept-connect1/socket-accept-connect1.cc

  Log Message:
  -----------
  Don't build promiscuous-sockets1 without uinet.


  Commit: 448ac46a069c433432f5515d0eebbfa71fb578bd
      https://github.com/wanproxy/wanproxy/commit/448ac46a069c433432f5515d0eebbfa71fb578bd
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/event_callback.h
    M event/typed_callback.h
    M http/http_server.cc
    M http/http_server.h
    M io/io_system.h
    M io/io_system_handle.cc
    M io/net/tcp_client.cc
    M io/net/tcp_client.h
    M io/net/udp_client.cc
    M io/net/udp_client.h
    M io/pipe/pipe_link.cc
    M io/pipe/pipe_link.h
    M io/pipe/pipe_producer.cc
    M io/pipe/pipe_splice.cc
    M io/pipe/pipe_splice.h
    M io/pipe/splice.cc
    M io/pipe/splice.h
    M io/pipe/splice_pair.cc
    M io/pipe/splice_pair.h
    M io/socket/socket_handle.cc
    M io/socket/socket_handle.h
    M programs/wanproxy/proxy_connector.cc
    M programs/wanproxy/proxy_connector.h
    M programs/wanproxy/proxy_socks_connection.cc
    M programs/wanproxy/proxy_socks_connection.h
    M programs/wanproxy/ssh_proxy_connector.cc
    M programs/wanproxy/ssh_proxy_connector.h
    M programs/wanproxy/ssh_stream.cc
    M programs/wanproxy/ssh_stream.h
    M ssh/ssh_transport_pipe.cc

  Log Message:
  -----------
  Checkpoint persistent callbacks II.

This converts EventCallback callbacks to be persistent callbacks,
as was done for SimpleCallback.

The only interesting difference over SimpleCallback is that we have
to be careful to steal the parameter to a TypedCallback and reset
it before executing the callback, or we can't reschedule the callback
from within its execution, which is a common pattern when two classes
are moving data between each other.


  Commit: e9c14537231675bdbd14c957cc48f6724d0591b4
      https://github.com/wanproxy/wanproxy/commit/e9c14537231675bdbd14c957cc48f6724d0591b4
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M event/typed_pair_callback.h
    M http/http_server.cc
    M http/http_server.h
    M io/net/tcp_client.cc
    M io/net/udp_client.cc
    M io/socket/simple_server.h
    M programs/wanproxy/proxy_connector.cc
    M programs/wanproxy/proxy_connector.h
    M programs/wanproxy/ssh_proxy_connector.cc
    M programs/wanproxy/ssh_proxy_connector.h

  Log Message:
  -----------
  Checkpoint persistent callbacks III.

Convert TypedPairCallback like TypedCallback to persist.


  Commit: 0b9fd94a8c52af770da6e6d21b80fa9db92b7d10
      https://github.com/wanproxy/wanproxy/commit/0b9fd94a8c52af770da6e6d21b80fa9db92b7d10
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-27 (Wed, 27 Jan 2016)

  Changed paths:
    M TODO.MT

  Log Message:
  -----------
  Update multithreading to-do.


  Commit: 24bddec9d8e7e13536ee989860180fa12441ead5
      https://github.com/wanproxy/wanproxy/commit/24bddec9d8e7e13536ee989860180fa12441ead5
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-28 (Thu, 28 Jan 2016)

  Changed paths:
    M event/callback.cc
    M event/callback.h
    M event/event_system.h

  Log Message:
  -----------
  Establish the scheduler in CallbackBase's constructor.

There's no need to wait until we schedule a callback to set its
scheduler.


  Commit: ccdab7fb8673299cdb4a5f8d99e90fc61b6507a6
      https://github.com/wanproxy/wanproxy/commit/ccdab7fb8673299cdb4a5f8d99e90fc61b6507a6
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-28 (Thu, 28 Jan 2016)

  Changed paths:
    M io/io_system.h
    M io/io_system_handle.cc
    M io/pipe/splice.cc
    M io/pipe/splice.h

  Log Message:
  -----------
  Don't cache schedulers by hand.

Schedulers now persist within the callback itself, so the scheduler just
needs to be available at the time that the callback itself is created.


  Commit: 46e25a2a836c2f339ad9b54f96c993921c79489b
      https://github.com/wanproxy/wanproxy/commit/46e25a2a836c2f339ad9b54f96c993921c79489b
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-28 (Thu, 28 Jan 2016)

  Changed paths:
    R NOTES.MT
    M crypto/crypto_encryption.h
    M crypto/crypto_encryption_openssl.cc
    M crypto/crypto_hash.h
    M crypto/crypto_hash_openssl.cc
    M crypto/crypto_mac.h
    M crypto/crypto_mac_openssl.cc
    M crypto/crypto_random.h
    M crypto/crypto_random_openssl.cc
    M event/event.h
    M event/event_callback.h
    M io/block_handle.cc
    M io/block_handle.h
    M io/channel.h
    M io/io_system.cc
    M io/io_system.h
    M io/io_system_handle.cc
    M io/net/udp_server.h
    M io/pipe/pipe.h
    M io/pipe/pipe_link.cc
    M io/pipe/pipe_link.h
    M io/pipe/pipe_producer.cc
    M io/pipe/pipe_producer.h
    M io/pipe/pipe_splice.cc
    M io/pipe/pipe_splice.h
    M io/pipe/splice.cc
    M io/pipe/splice.h
    M io/pipe/splice_pair.cc
    M io/stream_handle.cc
    M io/stream_handle.h
    M programs/wanproxy/proxy_socks_connection.cc
    M programs/wanproxy/proxy_socks_connection.h
    M programs/wanproxy/ssh_stream.cc
    M programs/wanproxy/ssh_stream.h
    M ssh/ssh_transport_pipe.cc
    M ssh/ssh_transport_pipe.h

  Log Message:
  -----------
  Get rid of the Buffer in Event.

This creates a new BufferEventCallback for the few kinds of events
that actually move out data.  Most kinds of events and over half of
the events we execute in practice don't move data with the event.
Only reads and their ilk do.

This also highlights the need to pass a reference to the Buffer,
not a copy of it, so that our refcount is low, and we can modify
the BufferSegments rather than keeping around a pristine copy on
the stack in the callback execution path.  Ugh.


  Commit: 6a0ef7eb5878b50f1a7c389c13e4fdf0d91e043f
      https://github.com/wanproxy/wanproxy/commit/6a0ef7eb5878b50f1a7c389c13e4fdf0d91e043f
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-28 (Thu, 28 Jan 2016)

  Changed paths:
    M TODO
    R TODO.MT

  Log Message:
  -----------
  Merge TODO.MT into TODO.

And while here remove wrong-headed and complete things from TODO.


  Commit: 7511fb4cdbaec9e7a98633189f3c04da6eb924fb
      https://github.com/wanproxy/wanproxy/commit/7511fb4cdbaec9e7a98633189f3c04da6eb924fb
  Author: Juli Mallett <juli at clockworksquid.com>
  Date:   2016-01-28 (Thu, 28 Jan 2016)

  Changed paths:
    M COPYRIGHT
    M programs/wanproxy/wanproxy.cc

  Log Message:
  -----------
  Bump primary copyright years.


Compare: https://github.com/wanproxy/wanproxy/compare/ae807c328102...7511fb4cdbae


More information about the wanproxy mailing list