Extraction with lift-json and Android

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Extraction with lift-json and Android

Nolan Darilek
Hello. Aside from the various cryptic symbols for method names, I'm really liking Dispatch so far. I'm having a bit of an issue using its JSON parsing in an Android project, however.

I've read that the lift-json library is best for Android apps, so I'm using dispatch-lift-json. I've also read that the Android environment in general doesn't play well at all with extraction to case classes. I think that there's a way to extract JSON values neatly, but I've read the code and can't figure it out. I've also checked out the Snapup app. I think that what would really help me is a simple example that isn't abstracted away into traits and such. Here's what I'm doing.

My app posts GPS parameters to a web service, getting back a JSON structure with strings representing displayable values. I want to encapsulate that structure in a case class. Currently I do it like so:

    val req = apiRoot/session/"position.json" << params
    http(req ># { response =>
      val f = Fix(
        (response \ "accuracy").extract[String],
        (response \ "altitude").extract[String],
        (response \ "direction").extract[String],
        (response \ "intersection") match {
          case JNothing => None
          case v => Some(v.extract[String])
        },
        (response \ "speed").extract[String],
        (response \ "way") match {
          case JNothing => None
          case v => Some(v.extract[String])
        }
      )

Is there a cleaner way to do that?

Also, I have portions of my JSON response that are lists of other JSON objects. Is this case handled at all? I read recently on the Lift list that lists and maps couldn't be the root of JSON trees until the library is modified to be completely 2.8-compliant, but I don't know if this extraction method considers subelements as the root when they're extracted.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Extraction with lift-json and Android

n8han
Administrator
On 6/15/10 6:19 PM, Nolan Darilek [via Databinder] wrote:
> I've read that the lift-json library is best for Android apps, so I'm
> using dispatch-lift-json. I've also read that the Android environment
> in general doesn't play well at all with extraction to case classes. I
> think that there's a way to extract JSON values neatly, but I've read
> the code and can't figure it out. I've also checked out the Snapup app.

Sure. With Snapup the json processing is all factored up into the Meetup
interface. [1] Generally you can do this adhoc, or you can write an
interface for as much of the web API as you need for your app to work.

[1]:
http://sourced.implicit.ly/net.databinder/dispatch-meetup/0.7.4/Meetup.scala.html

> I think that what would really help me is a simple example that isn't
> abstracted away into traits and such. Here's what I'm doing.

There are some examples of ad hoc processing in lift-json's tests.
Generally, you work with an iterator that traverses the entire
structure. The problem I have with that technique is I'm usually very
concerned with the location of properties in the structure. I only want
"id" if it's at the top level, or nested in exactly one way, and I
didn't find a convenient way to express that in what comes with
lift-json *except* the case class extraction, which relies on ParaNamer
and that isn't going to work on Android as far as I can tell. So...

> My app posts GPS parameters to a web service, getting back a JSON
> structure with strings representing displayable values. I want to
> encapsulate that structure in a case class.

I don't use case classes for this, for reasons I get into below. Instead
I use higher order functions to describe each element I want from the
json and use to those project values as I need them. If the value exists
of the correct type, I get it back in a list. If it doesn't, I get Nil.
This also encourages you to write code that can handle some drift in the
web API.

You can see this in the Meetup interface. You may not like its use of
symbolic method names, but for me it was most important to be able to
define these interfaces with minimal cruft. If you want to do something
one off, you could just define the interface for the parts to need. I'll
try to do it here for the your case.

object Fix {
   val accuracy = 'accuracy ? str
   val altitude = 'altitude ? str
   val direction = 'direction ? str
   val intersection = 'intersection ? str
   val speed = 'speed ? str
   val way = 'way ? str
}

Then you use that to get values as they need them, and there is no
difference between the values that you expect to possibly be null or not
there and those you do. They're all List[String]. (Could have been
Option[String], but List was more flexible for also dealing with arrays.)

> [...]
> Also, I have portions of my JSON response that are lists of other JSON
> objects. Is this case handled at all? I read recently on the Lift list
> that lists and maps couldn't be the root of JSON trees until the
> library is modified to be completely 2.8-compliant, but I don't know
> if this extraction method considers subelements as the root when
> they're extracted.

It seems that there are a lot of lurking problems with using case
classes to model json. I've also hit the maximum number of constructor
parameters for a json object with a lot of properties. When you hit a
dead end like that. you you have to start over and rebuild your model.
So, personally, I'm going to avoid it.

Nathan

Reply | Threaded
Open this post in threaded view
|

Re: Extraction with lift-json and Android

Nolan Darilek
OK, I think I'm still confused. Sorry, I'm looking through examples and specs, but I'm just not able to apply those to my simple case.

So given the example for Fix, with higher-order extraction functions, how do I then extract values from a web service call in a useful way? Given that I'm doing:

    http(req ># { response =>
      val f = Fix(
        ...
      )
      processFix(f)

Do I then need to make processFix() take a bunch of individual parameters? Or is there a cleaner way? Do I just pass it my JSON response and call Fix.intersection, Fix.way, etc.?

Also, at the moment I'm only extracting surface-level parameters from fixes, but I'd also like to extract values of the form:

"nearby":[
  {"name":"Gas station","direction":"ahead","distance":"100 meters'}
]

Given that as a subelement of fixes, how would I extract that? In my web service, those values are called relations, so I'd imagine I'd do:

object Relation {
  val name = 'name ? str
  val direction = 'direction ? str
  val distance = 'distance ? str
}

but I'm not immediately sure how to go from:

object Fix {
  val nearby = 'nearby ? ary
}

to an array of relations. Is this doable?

Also, why vals and not defs for the functions?

I'm looking through the meetup example, specs and such, vut a) I'm not familiar with the Meetup API and b) there's not much documentation. I think that what might help is a bit more clarification in the stdout examples, or maybe a bit more fleshing out of the common tasks section.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Extraction with lift-json and Android

Nolan Darilek
OK, after a good bit of trial and error, I think I've figured it out. I'll include my results here for anyone who, like me, was googling for an answer.

I rewrote Fix to look like so:

class Fix(json:JValue) {
  lazy val accuracy = ('accuracy ? str)(json).head
  lazy val altitude = ('altitude ? str)(json).head
  lazy val direction = ('direction ? str)(json).head
  lazy val intersection = ('intersection ? str)(json)
  lazy val nearbyPoints = ('nearby ? ary)(json).map(new Relation(_))
  lazy val nearestPoints = ('nearest ? ary)(json).map(new Relation(_))
  lazy val speed = ('speed ? str)(json).head
  lazy val way = ('way ? str)(json)
}

class Relation(json:JValue) {
  lazy val name = ('name ? str)(json).head
  lazy val direction = ('direction ? str)(json).head
  lazy val distance = ('distance ? str)(json).head

  override lazy val toString = name+": "+distance+" "+direction
}

To answer my own question, the lazy vals ensure that the value isn't re-extracted on each call. My request now looks like:

    http(req ># { response =>
      val f = new Fix(response)
      processFix(f)
    }

    And to access various Fix parameters:

        f.way.foreach { w => sendMessage("On "+w) }
        f.intersection.foreach { i => sendMessage(i, IntersectionType) }

Those are the only two optional return values that a Fix can have, which is why some of the values use .head automatically. I guess if I didn't have control over the web service, that might not be the best option.

Out of curiosity, I still don't get why Options aren't used. I could have still run foreach on the Option class, so I'm not sure what benefit I get from treating return values like lists of a single item or Nil. What am I missing?

Thanks again.
Reply | Threaded
Open this post in threaded view
|

Re: Extraction with lift-json and Android

n8han
Administrator
On 6/16/10 5:41 PM, Nolan Darilek [via Databinder] wrote:
> OK, after a good bit of trial and error, I think I've figured it out.
> I'll include my results here for anyone who, like me, was googling for
> an answer.
>
> I rewrote Fix to look like so:

Very interesting! That's different from how I've used it, but in my
cases I didn't need to reuse the values a lot. So I would just pass
around the JValue and apply the particular extractor when I needed it.

> To answer my own question, the lazy vals ensure that the value isn't
> re-extracted on each call.

Right, although in my case the functions hadn't been applied at all. But
a val at least avoids having to recompose the function on each call.

> Out of curiosity, I still don't get why Options aren't used. I could
> have still run foreach on the Option class, so I'm not sure what
> benefit I get from treating return values like lists of a single item
> or Nil. What am I missing?

It had to with using flatMap on a series of extractors. Initially I
typed single properties like str as an Option, but then I had problems
using them in combination with ary. (My hunch at the time was that 2.8
would not have the same issue with mixing Option and List, but I haven't
tested that.) Combining ary with simple properties, you can do some
pretty interesting stuff:

val questions: JValue => List[String] = 'questions ? ary >>~> str

which is just an alias for

'questions ? ary andThen { _ flatmap str }

That just naturally returns a list of strings, ignoring any element that
isn't a string or returning Nil if there is no list named "questions" in
the first place.

This is all oriented towards delaying the extraction operation. If as in
your case you don't care to do that, you don't need so much "andThen" as
you can just apply things at the first stage. Someone could come up with
a more suitable interface for that scenario. Still, I think this one is
not so bad for it. :)

Nathan