huge difference using wanproxy throught internet

Fri Aug 16 14:25:30 PDT 2013

2013/8/16 Juli Mallett <juli at clockworksquid.com>

> Hi Daniel,
>
> With a lower-latency link like a WAN rather than the Internet, I would
> expect to see more packets sent between WANProxy instances; WANProxy makes
> no effort to buffer up data before sending it.  Using a slower link, you'll
> naturally have more buffering occurring and fewer packets sent.  There's
> some things that we could do and should do to avoid sending out gratuitous
> small packets, and I have some changes to do that, but I haven't tested
> them enough to check them in yet.
>
> I'm not sure if I understand the difference you're seeing in terms of data
> reduction / deduplication / compression.  Which is giving the greater
> savings in terms of data transferred, the LAN or the Internet?
>

The LAN, definitely. the savings are enormous. In a 168Mb file transfer the
first time there's 172Mb of traffic between WANProxy instances, the second
time it doesn't even reach 2Mb. of data transferring.
Transfer time follows a similar pattern, 14 seconds the first download, 4.2
seconds the second one.

>  And what kind of download time are you seeing to the client on each
> configuration?
>

The transfer time evaluation is tricky using Internet because the link
speed is not constant, well the link is but the links between the routers I
jump to reach to the ending point might not. That's why I took a closer
look in terms of data/pkts transfer.
Nevertheless the transfer time difference between first and second file
transfer is not substantial.

I dug a bit deeper and I found that what seems to be causing the problem is
not the Internet o MTU, but http-proxy I use behind the WANProxy "server".
If I change the setup from this:
WPclient->(gateway)->//internet//->WPserver+squid->//internet//->webserver
to this:
WPclient->(gateway)->//internet//->WPserver->webserver

and download the very same file, it worked like a charm. the second time
the file downloaded a LOT faster.
This time instead of being at some server on the internet I put it in a
local webserver that lives in the same server as the ending point of
WANProxy.
But I can't figure out why this is causing problems, and huge ones.

downloading the same file twice using the first setup takes 10 to 20% less
than the first time, when using the second setup the file is downloaded 60
to 70% faster at the second time.

If I use the ending http-proxy locally (that is by connecting via ssh to
the server and using the local http proxy to download this very same file
from the internet) the transfering is very fast. The internet connection of
this server is a lot faster that the one I have at the gateway of my LAN.

Could it be that the communication between WANProxy end point with a http
proxy creates too many packets between them, and this problem gets somehow
translated to the WAProxy starting point "the client"?

I hope I explained myself, if not I'll try to send some "drawings" to
explain the problem better. Please excuse my english.

thanks for any tips.

dc//

>
> Thanks,
> Juli.
>
>
> On Fri, Aug 16, 2013 at 8:25 AM, Daniel Coletti <dcoletti at xtech.com.ar>wrote:
>
>> Hi,
>>     We're testing wanproxy heavily and during the tests we saw a
>> baffling traffic saving difference between using wanproxy inside a LAN and
>> using it trought the internet.
>>
>> Under Linux (ubuntu 12.04 client side and debian 7.0 on the server).
>> The first set up is like this (we used an iso file for testing):
>> WPclient->(gateway)->//internet//->WPserver+squid->//internet//->webserver
>>
>> 1st. download:
>> traffic between wanproxies = 218.000 tcp packs, 183 mbytes
>> traffic between wanproxy client and PC inside the LAN = 29.412 tcp packs,
>> 177Mb
>>
>>  2nd download:
>> about 10% traffic saving equally disperse between tcp packs and mbytes
>> sent between wanproxies
>>
>> the second set up is like this (the same iso file):
>> WPcliente->(LAN)WPserver+squid->(LAN)->webserver
>> 1st. download:
>> traffic between wanproxies = 104.000 tcp packs, 143 mbytes
>> traffic between wanproxy client and PC inside the LAN = 7844 tcp packs,
>> 177Mb
>>
>> 2nd download:
>> traffic between wanproxies = 8230 tcp packs, *10* mbytes
>> traffic between wanproxy client and PC inside the LAN = 4090 tcp packs,
>> 176Mb
>>
>> The first thing I notice was the huge difference in the tcp packs sent
>> during the first download of both setups (Internet vs. LAN) 218K vs 104K,
>> but the differences between the first and second download using LAN is 20%
>> saving using internet, 95% saving using LAN.
>>
>> Following Diego Woitasen's advice I tried manipuling the MTU with MSS
>> Clamping (http://lartc.org/howto/lartc.cookbook.mtu-mss.html) and it got
>> better (up to 30% savings).
>>
>> Using internet the number of tcp packets sent between wanproxies grew
>> from 104K  (in LAN) to 218K (more than 100%).
>>
>> Any ideas on how to go around this problem? Is there something that can
>> be done inside wanproxy's code?
>>
>> thanks,
>>
>> dc//
>>
>> --
>> Daniel Coletti
>> Director
>> XTech (Soluciones Linux para Empresas) - http://www.xtech.com.ar
>> ++(5411) 5219-0678
>> 25 de Mayo 460 - 3er. piso
>> Buenos Aires, Argentina
>>
>> _______________________________________________
>> wanproxy mailing list
>> wanproxy at lists.wanproxy.org
>> http://lists.wanproxy.org/listinfo.cgi/wanproxy-wanproxy.org
>>
>>
>

-- 
Daniel Coletti
Director
XTech (Soluciones Linux para Empresas) - http://www.xtech.com.ar
++(5411) 5219-0678
25 de Mayo 460 - 3er. piso
Buenos Aires, Argentina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wanproxy.org/pipermail/wanproxy-wanproxy.org/attachments/20130816/cfe5499e/attachment-0002.htm>