Using httpclient-cache

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Using httpclient-cache

Alex Tucker
Hi,

Now that httpclient 4.1 is out with support for HTTP caching in httpclient-cache, I wanted to use it via your dispatch-http library.

However, I can't figure out a way to override client in dispatch.Http in a sensible way.  Apache's new CachingHttpClient uses the decorator pattern and wraps an underlying Apache HttpClient implementation.  

While dispatch.ConfiguredHttpClient extends HttpClient, so I should just be able to wrap it in a CachingHttpClient, the use in dispatch.Http is specific to ConfiguredHttpClient, through the use of credentials as far as I can see.

Am I just being daft, or would you need to rewire ConfiguredHttpClient and its use in Http a little?

Alex.
Reply | Threaded
Open this post in threaded view
|

Re: Using httpclient-cache

n8han
Administrator
On 02/04/2011 12:04 PM, Alex Tucker [via Databinder] wrote:
> Now that httpclient 4.1 is out with support for HTTP caching in
> httpclient-cache, I wanted to use it via your dispatch-http library.

Hi Alex. The 4.1 client is going to be used in Dispatch 0.8.x and I've
been working with the betas and alphas of it on the 080 branch for a
while. There's a lot of good stuff in there, like Google AppEngine
support, and more recently, an NIO client. I had to refactor the hell
out of everything to make that work, but it's all for the better.

> However, I can't figure out a way to override client in dispatch.Http
> in a sensible way.  Apache's new CachingHttpClient uses the decorator
> pattern and wraps an underlying Apache HttpClient implementation.
>
> While dispatch.ConfiguredHttpClient extends HttpClient, so I should
> just be able to wrap it in a CachingHttpClient, the use in
> dispatch.Http is specific to ConfiguredHttpClient, through the use of
> credentials as far as I can see.

You should be able to do that on 0.8.x as far as I can tell.

def http = new Http {
     override def make_client = new CachingHttpClient(super.make_client)
}

To do this you'll need to check out the 080 branch from github. I'm also
very close to releasing a new beta. It seems to work well; but it would
be great if someone else could bang around on the branch a bit to double
check. Other than that, the main hangup is just documenting everything I
changed since the last beta. I don't even know where to start. :\

Nathan
Reply | Threaded
Open this post in threaded view
|

Re: Using httpclient-cache

daggerrz
I'm trying to do the same thing. Unfortunately, make_client in BlockingHttp is inferred to the ConfiguredHttpClient class (of Dispatch) instead of the HttpClient interface from Apache (which renders it impossible to use CachingHttpClient).

The problem becomes visible at compile time if the return type of make_client is set:

  def make_client : HttpClient = new ConfiguredHttpClient

I've fixed this (although not elegantly) in my fork at https://github.com/daggerrz/Databinder-Dispatch/commit/417a1fff5866a97783b89b6d5b8ceb2218c80071.

This allows for mixing a cache trait into Http like this:

  trait HttpCache extends Http {
    abstract override def make_client : org.apache.http.client.HttpClient = {
      val config = new CacheConfig
      config.setMaxCacheEntries(500)
      config.setMaxObjectSizeBytes(50000) // 25 MB cache
      new CachingHttpClient(super.make_client, config)
    }
  }

Dag
Reply | Threaded
Open this post in threaded view
|

Re: Using httpclient-cache

n8han
Administrator
On 03/04/2011 05:05 PM, daggerrz [via Databinder] wrote:
> I'm trying to do the same thing. Unfortunately, make_client in
> BlockingHttp is inferred to the ConfiguredHttpClient class (of
> Dispatch) instead of the HttpClient interface from Apache (which
> renders it impossible to use CachingHttpClient).

Oh, that's not good. It's too bad we have to dance around with
DynamicVariables in the first place. I never understood why HttpClient
thinks of credentials as being global to a client, not particular to a
request. Maybe that will change in the async version, which so far
doesn't seem to have any opinion/implementation for credentials.

I think it's best not to impose any restriction on the client type
beyond it being an HttpClient, so now BlockingHttp has the credentials
reference and passes it to ConfiguredHttpClient and its subclasses. This
is going to be another breaking change, may as well get them all out of
the way!

I'll try to release this as the next beta later tonight, since people
seem to be keen on getting the caching working. I guess that means it
would probably make a good submodule too.

Nathan
Reply | Threaded
Open this post in threaded view
|

Re: Using httpclient-cache

daggerrz
Just confirming that this indeed works for us. Thanks!