Help understanding twitter client

[Jared Casper]

So for better or worse I'm using learning/using dispatch as a catalyst for learning Scala, so I hope you'll forgive a naive question or two.  I'm not really interested in the Twitter client itself, but am trying to understand it so that I can write clients for other things.

My question is how to use the dispatch.twitter.Search object effectively.  I get how the SearchBuilder works in creating a request/handler object ready to be executed.  Executing it gives me a (HttpPackage wrapping a) List of JsObjects.  What I don't see, and this is probably my lack Scala experience/knowledge, is how to use the extractors contained in the Search object to easily get at the data in the List of JsObjects.  I can get at the fields individually (i.e. "res map { _ match { case Search.from_user(u) => u } }" to get a List of the users).  But is there way to extract all of the fields together into an object?  Or how are those extractors typically used?  Is there an example use of the Twitter client beyond the very simple spec test (i.e. uses the Search, Account, User, etc. objects)

Or maybe a more general question: is there any docs/examples of using the native dispatch json package?  Is there any reason to use it over the lift json package?  In any case, I'd like to understand fully how it works to further my scala knowledge.

Thanks for any direction.

[n8han]

On 07/25/2011 10:09 PM, Jared Casper [via Databinder] wrote:

> My question is how to use the dispatch.twitter.Search object
> effectively.  I get how the SearchBuilder works in creating a
> request/handler object ready to be executed.  Executing it gives me a
> (HttpPackage wrapping a) List of JsObjects.  What I don't see, and
> this is probably my lack Scala experience/knowledge, is how to use the
> extractors contained in the Search object to easily get at the data in
> the List of JsObjects.  I can get at the fields individually (i.e.
> "res map { _ match { case Search.from_user(u) => u } }" to get a List
> of the users).  But is there way to extract all of the fields together
> into an object?

Lift-json can deserialize into case classes, as can a few other
libraries. I'm not a big fan of this method when used with APIs outside
your control, as it assumes that the API will forever conform to what
you've defined in the case class, and what case classes in general can
represent. For example, there is an upper bound on the number of fields
a case class can have in Scala (the number of parameters its constructor
can have), and JSON does not share this constraint. Nulls are also
problematic; will your class use Options everywhere, or assume that
nulls or missing fields do not occur?

So instead I tried to come up with ways to succinctly define the
expected object-path to and type of a value, and if it's not there in
that format then it's None or Nil. The dispatch-http-json module was the
first pass at that.

> Or how are those extractors typically used?  Is there an example use
> of the Twitter client beyond the very simple spec test (i.e. uses the
> Search, Account, User, etc. objects)

The only other example I've done is the twine app, but it just uses
lift-json ad-hoc to get at the fields it needs.

> Or maybe a more general question: is there any docs/examples of using
> the native dispatch json package?  Is there any reason to use it over
> the lift json package?  In any case, I'd like to understand fully how
> it works to further my scala knowledge.

There really isn't a reason to use dispatch-json, now. If you want to
use that kind of parser you should use Spray [1], which is apparently
based on it. I switched over to lift-json myself when I found that
parser-combinators (such as dispatch-json uses) didn't work on Android.

At that point I took a second pass at defining terms to create schemata
for json API results, which you can find in the dispatch-lift-json
module. The Meetup API integration has a extensive definitions in this
form [2] and you can even find them productively used in the Snapup
Android app [3].

I think these experiments showed some promise but ultimately the
interaction is just to *weird*, as it stands. Even I have a hard time
remembering how they work. I want a more natural way of exactly what
they do--enabling the static definition of what is expected from a json
structure, which fails softly when the data does not conform. Since json
traversal is such a big deal I assume that someone will solve this
properly at some point, and then I can just offer a binding for this in
Dispatch.

[1]: https://github.com/spray/spray-json
[2]:
https://github.com/n8han/dispatch-meetup/blob/master/src/main/scala/Meetup.scala#L92
[3]:
https://github.com/meetup/snapup-android/blob/master/src/main/scala/Meetups.scala#L40

Nathan

[eed3si9n]

- [Weaving tweed with Scala and Json](http://technically.us/code/x/weaving-tweed-with-scala-and-json/)

- [twine](https://github.com/n8han/dispatch-twine/)

-eugene

On Mon, Jul 25, 2011 at 10:09 PM, Jared Casper [via Databinder] <[hidden email]> wrote:

So for better or worse I'm using learning/using dispatch as a catalyst for learning Scala, so I hope you'll forgive a naive question or two.  I'm not really interested in the Twitter client itself, but am trying to understand it so that I can write clients for other things.

My question is how to use the dispatch.twitter.Search object effectively.  I get how the SearchBuilder works in creating a request/handler object ready to be executed.  Executing it gives me a (HttpPackage wrapping a) List of JsObjects.  What I don't see, and this is probably my lack Scala experience/knowledge, is how to use the extractors contained in the Search object to easily get at the data in the List of JsObjects.  I can get at the fields individually (i.e. "res map { _ match { case Search.from_user(u) => u } }" to get a List of the users).  But is there way to extract all of the fields together into an object?  Or how are those extractors typically used?  Is there an example use of the Twitter client beyond the very simple spec test (i.e. uses the Search, Account, User, etc. objects)

Or maybe a more general question: is there any docs/examples of using the native dispatch json package?  Is there any reason to use it over the lift json package?  In any case, I'd like to understand fully how it works to further my scala knowledge.

Thanks for any direction.

---

If you reply to this email, your message will be added to the discussion below:

http://databinder.3617998.n2.nabble.com/Help-understanding-twitter-client-tp6620785p6620785.html

To start a new topic under Dispatch, email [hidden email]
To unsubscribe from Dispatch, click here.

[Jared Casper]

On Mon, Jul 25, 2011 at 7:48 PM, eed3si9n [via Databinder]
<[hidden email]> wrote:
> - [Weaving tweed with Scala and
> Json](http://technically.us/code/x/weaving-tweed-with-scala-and-json/)

Ah, lots of good stuff there.  Thanks!

Jared

[Jared Casper]

On Mon, Jul 25, 2011 at 7:47 PM, n8han [via Databinder]
<[hidden email]> wrote:
...
> I think these experiments showed some promise but ultimately the
> interaction is just to *weird*, as it stands. Even I have a hard time
> remembering how they work. I want a more natural way of exactly what
> they do--enabling the static definition of what is expected from a json
> structure, which fails softly when the data does not conform. Since json
> traversal is such a big deal I assume that someone will solve this
> properly at some point, and then I can just offer a binding for this in
> Dispatch.
>

Very interesting.  Thanks for the in-depth reply.

Jared