persistent cache

[Diego Woitasen]

Yes, I saw your changes. It's a good starting point for anyone that
want to implement a new cache.

I'll post any news on the mailing list if I improve my code.

Regards,
 Diego

On Mon, Nov 21, 2011 at 5:36 PM, Juli Mallett <juli at clockworksquid.com> wrote:
> On Mon, Nov 21, 2011 at 09:17, Diego Woitasen <diego at woitasen.com.ar> wrote:
>> https://github.com/diegows/wanproxy
>>
>> There is, it's an ugly hack, far from a definitive real implementation
>> but works. It doesn't support ageing yet. It's was developed to test
>> the performance and to demostrate to one of my clients that it's not a
>> 1 year of development. I coded this in a few hours.
>>
>> I think because Wanproxy is modular, we can have more than one
>> on-disk-cache type. :)
>
> You're exactly right that we can support multiple, and I was thinking
> about including your bdb code quite seriously, or something like it,
> as a stopgap.  I'd first need to handle the LRU (or whatever) problem,
> getting rid of unused segments with no outstanding references, and,
> even harder, exerting some kind of pressure if all of them are in use
> and a limit has been reached.  I know how I want to handle making sure
> that we don't send references to data by hash and then expunge the
> segment from our cache, I think, but that's not trivial, too.
>
> Have you seen the changes I recently made to tack and to xcodec_cache?
>  I added what I hope are some helpful hooks for you and added examples
> to tack of (1) using an on-disk cache [only suitable for tack, not
> WANProxy, unless you only care about persistence and not paging] (2)
> using no cache except the backref cache, which might be useful in
> WANProxy for people who want to do some light deduplication on data
> before sending it to zlib, but who don't want memory usage or any
> added latency because of the need to send data back and forth to
> answer unresolved references, etc.
>
>> I'll try to implement this correctly with Bdb when I have more time.
>>
>> Regards,
>>  Diego
>>
>> On Fri, Nov 18, 2011 at 11:42 PM, Juli Mallett <juli at clockworksquid.com> wrote:
>>> On Fri, Nov 18, 2011 at 18:15, Eric Hester <eric.hester at gmail.com> wrote:
>>>>
>>>> > How far away is the persistent cache feature?
>>>>
>>>
>>> diegows has an experimental version using bdb, which is non-ideal, but good
>>> enough for some deployments.  I unfortunately still don't really have the
>>> time to see to completion in the near future my own effort.
>>> If you google for wanproxy bdb you should be able to find Diego's github
>>> tree, I think.
>>> Thanks,
>>> Juli.
>>>
>>> _______________________________________________
>>> wanproxy mailing list
>>> wanproxy at lists.wanproxy.org
>>> http://lists.wanproxy.org/listinfo.cgi/wanproxy-wanproxy.org
>>>
>>>
>>
>>
>>
>> --
>> Diego Woitasen
>>
>



-- 
Diego Woitasen