Simple CRUD example for comparison with other frameworks and toolkits

[laufer]

Hi, I am a computer science researcher and educator who teaches web services regularly (every three semesters usually).

http://laufer.cs.luc.edu/teaching/433

I also contributed some example code to Restlet and participate in the implementing-rest project, where I recently added entries for Scala and Haskell.

http://code.google.com/p/implementing-rest/ 

Among the various frameworks and toolkits that support Scala, I am intrigued by Unfiltered because of the elegance it derives from pattern matching and higher order functions, without obscuring from my students how a service works.

For comparison with other approaches, especially Restlet, I have rigged up a simple but fairly complete example of a web service with two levels of CRUD and authentication. It is based on my version of unfiltered-basic-auth.g8, which I forked so I could upgrade to 0.3.1. It is intended to be fully compatible (observationally equivalent) with the Restlet version of the bookmarks example from the RESTful Web Services book by Richardson and Ruby.

https://github.com/webservices-cs-luc-edu/unfiltered-example-bookmarks

Please critique it if you have time. I plan to work on this and other examples as my schedule permits. It also serves as a context for trying out ideas, one of which I will float shortly in a separate post.

Thank you,

Konstantin

[n8han]

On 03/10/2011 04:56 PM, laufer [via Databinder] wrote:
> Among the various frameworks and toolkits that support Scala, I am
> intrigued by Unfiltered because of the elegance it derives from
> pattern matching and higher order functions, without obscuring from my
> students how a service works.

Thanks, that's very good to hear.

> https://github.com/webservices-cs-luc-edu/unfiltered-example-bookmarks
>
> Please critique it if you have time. I plan to work on this and other
> examples as my schedule permits. It also serves as a context for
> trying out ideas, one of which I will float shortly in a separate post.

I encourage anyone and everyone on the list to jump in here. We don't
really have idioms yet in Unfiltered, though I like to think broader
functional idioms apply most of the time; it would certainly be an asset
to have an agreed-upon solid design for a common class of applications.

Nathan

[max]

On Thu, Mar 10, 2011 at 17:02, n8han [via Databinder]
<[hidden email]> wrote:
>> Please critique it if you have time. I plan to work on this and other
>> examples as my schedule permits. It also serves as a context for
>> trying out ideas, one of which I will float shortly in a separate post.
>
> I encourage anyone and everyone on the list to jump in here. We don't
> really have idioms yet in Unfiltered, though I like to think broader
> functional idioms apply most of the time; it would certainly be an asset
> to have an agreed-upon solid design for a common class of applications.

The gold is in those idioms. I absolutely *love* how the various plans
are initialized and organized. I actually wrote a commercial
Unfiltered application for work, and my plan code isn't anywhere as
clean as this. Well done.

[laufer]

max wrote

On Thu, Mar 10, 2011 at 17:02, n8han [via Databinder]
<[e-mail]> wrote:
>> Please critique it if you have time. I plan to work on this and other
>> examples as my schedule permits. It also serves as a context for
>> trying out ideas, one of which I will float shortly in a separate post.
>
> I encourage anyone and everyone on the list to jump in here. We don't
> really have idioms yet in Unfiltered, though I like to think broader
> functional idioms apply most of the time; it would certainly be an asset
> to have an agreed-upon solid design for a common class of applications.

The gold is in those idioms. I absolutely *love* how the various plans
are initialized and organized. I actually wrote a commercial
Unfiltered application for work, and my plan code isn't anywhere as
clean as this. Well done.

Wow, thanks a lot for these prompt and encouraging responses!

Btw, here is a brief post of mine about the "happy path" approach for successive pattern matching. I think it aids in keeping the plans clean.

http://blog.etl.luc.edu/post/3766880636

[Dustin Whitney]

The only big difference I can see between your implementation and how I do things is I usually use the cake pattern for dependency injection, but it's something that can be confusing at first and may detract from teaching someone about web services, so I think your approach is better given the context.

Dustin

On Thu, Mar 10, 2011 at 10:13 PM, laufer [via Databinder] <[hidden email]> wrote:

max wrote:

On Thu, Mar 10, 2011 at 17:02, n8han [via Databinder]

<[hidden email]> wrote:

>> Please critique it if you have time. I plan to work on this and other
>> examples as my schedule permits. It also serves as a context for
>> trying out ideas, one of which I will float shortly in a separate post.
>
> I encourage anyone and everyone on the list to jump in here. We don't
> really have idioms yet in Unfiltered, though I like to think broader
> functional idioms apply most of the time; it would certainly be an asset
> to have an agreed-upon solid design for a common class of applications.

The gold is in those idioms. I absolutely *love* how the various plans
are initialized and organized. I actually wrote a commercial
Unfiltered application for work, and my plan code isn't anywhere as
clean as this. Well done.

Wow, thanks a lot for these prompt and encouraging responses!

Btw, here is a brief post of mine about the "happy path" approach for successive pattern matching. I think it aids in keeping the plans clean.

http://blog.etl.luc.edu/post/3766880636

---

If you reply to this email, your message will be added to the discussion below:

http://databinder.3617998.n2.nabble.com/Simple-CRUD-example-for-comparison-with-other-frameworks-and-toolkits-tp6159509p6159547.html

To start a new topic under Unfiltered, email [hidden email]
To unsubscribe from Unfiltered, click here.

[laufer]

Dustin Whitney wrote

The only big difference I can see between your implementation and how I do
things is I usually use the cake pattern for dependency injection, but it's
something that can be confusing at first and may detract from teaching
someone about web services, so I think your approach is better given the
context.

Good point! The cake pattern has been on my list of things to learn for a while. I agree in that it's probably best to teach it to them separately, perhaps as a refactoring of this example.

Thanks,

Konstantin

[chris_lewis]

The structure is very nice indeed!

One thing the example seems to sidestep is parameter validation, which I've consistently been bothered by in terms of noise. I'd be interested in hearing how others deal with this, but the technique I've been using to have dedicated validation objects per plan. I keep the scope of my plans small and my validations have all been purely functional, moving them to a separate object seemed natural.

I've had another issue bugging me, but I think a different thread would suit.

-chris

[laufer]

chris_lewis wrote

The structure is very nice indeed!

One thing the example seems to sidestep is parameter validation, which I've consistently been bothered by in terms of noise. I'd be interested in hearing how others deal with this, but the technique I've been using to have dedicated validation objects per plan. I keep the scope of my plans small and my validations have all been purely functional, moving them to a separate object seemed natural.

I've had another issue bugging me, but I think a different thread would suit.

-chris

Thanks, Chris, excellent point. I will tackle validation some time soon.

Btw, I just corrected a very serious fundamental problem with the example. Perhaps nobody commented on it because it was orthogonal to Unfiltered: the repository was not thread-safe! The easiest fix seemed to convert its in-memory version into a fully synchronized object. Because I did not want to use Java synchronized, I created a decorator that transparently wraps an actor around it.

Konstantin

[Dustin Whitney]

I typically use STM to wrap immutable maps held in memory. I tend to use STM instead of actors, because there is no blocking with STM, and it's a little easier to use. Check out ScalaSTM http://nbronson.github.com/scala-stm/index.html In fact I think you're hard pressed to find examples where STM isn't a better solution than actors... maybe remoting, but I'm skeptical about that too.

D

On Mar 14, 2011 8:35pm, "laufer [via Databinder]" <[hidden email]> wrote:

>
> chris_lewis wrote:
>
> The structure is very nice indeed!
>
>
> One thing the example seems to sidestep is parameter validation, which I've consistently been bothered by in terms of noise. I'd be interested in hearing how others deal with this, but the technique I've been using to have dedicated validation objects per plan. I keep the scope of my plans small and my validations have all been purely functional, moving them to a separate object seemed natural.
>
>
> I've had another issue bugging me, but I think a different thread would suit.
>
>
> -chris
>
>
>
>
> Thanks, Chris, excellent point. I will tackle validation some time soon.
>
>
> Btw, I just corrected a very serious fundamental problem with the example. Perhaps nobody commented on it because it was orthogonal to Unfiltered: the repository was not thread-safe! The easiest fix seemed to convert its in-memory version into a fully synchronized object. Because I did not want to use Java synchronized, I created a decorator that transparently wraps an actor around it.
>
>
> Konstantin
>
>
>
>
>
>
>
> If you reply to this email, your message will be added to the discussion below:
>
> http://databinder.3617998.n2.nabble.com/Simple-CRUD-example-for-comparison-with-other-frameworks-and-toolkits-tp6159509p6170549.html
>
>
>
> To start a new topic under Unfiltered, email [hidden email]
>
> To unsubscribe from Unfiltered, click here.
>
>

[laufer]

Brilliant suggestion, thank you! I was not aware of STM for Scala. This will help me with various teaching and research explorations.

On Mon, Mar 14, 2011 at 20:00, Dustin Whitney [via Databinder] <[hidden email]> wrote:

I typically use STM to wrap immutable maps held in memory. I tend to use STM instead of actors, because there is no blocking with STM, and it's a little easier to use. Check out ScalaSTM http://nbronson.github.com/scala-stm/index.html In fact I think you're hard pressed to find examples where STM isn't a better solution than actors... maybe remoting, but I'm skeptical about that too.

[laufer]

I converted the decorators to traits and added an STM-based one. This feels very nice and natural now.

Besides, it looks like either Scala's actors are buggy or there is some interference when printing or using STM atomic in the react handler.

Thanks again!

laufer wrote

Brilliant suggestion, thank you! I was not aware of STM for Scala. This will
help me with various teaching and research explorations.

On Mon, Mar 14, 2011 at 20:00, Dustin Whitney [via Databinder] <
[e-mail]> wrote:

> I typically use STM to wrap immutable maps held in memory. I tend to use
> STM instead of actors, because there is no blocking with STM, and it's a
> little easier to use. Check out ScalaSTM
> http://nbronson.github.com/scala-stm/index.html In fact I think you're
> hard pressed to find examples where STM isn't a better solution than
> actors... maybe remoting, but I'm skeptical about that too.
>

[n8han]

I've finally read through the code, and I like it a lot. My only suggestion is pretty superficial, that it would be nice to have an extractor for your user verification so that you don't have to repeat the "case BasicAuth(u, p) if verify(u, p, user)" each time. Might be tricky though, with an abstract auth service.

I'll take another pass through it later and see if anything else jumps out at me.

Nathan

On 03/15/2011 05:09 PM, laufer [via Databinder] wrote:

I converted the decorators to traits and added an STM-based one. This feels very nice and natural now.

Besides, it looks like either Scala's actors are buggy or there is some interference when printing or using STM atomic in the react handler.

Thanks again!

laufer wrote:

Brilliant suggestion, thank you! I was not aware of STM for Scala. This will
help me with various teaching and research explorations.

On Mon, Mar 14, 2011 at 20:00, Dustin Whitney [via Databinder] <
[hidden email]> wrote:

> I typically use STM to wrap immutable maps held in memory. I tend to use
> STM instead of actors, because there is no blocking with STM, and it's a
> little easier to use. Check out ScalaSTM
> http://nbronson.github.com/scala-stm/index.html In fact I think you're
> hard pressed to find examples where STM isn't a better solution than
> actors... maybe remoting, but I'm skeptical about that too.
>

---

If you reply to this email, your message will be added to the discussion below:

http://databinder.3617998.n2.nabble.com/Simple-CRUD-example-for-comparison-with-other-frameworks-and-toolkits-tp6159509p6174326.html

To start a new topic under Unfiltered, email [hidden email]
To unsubscribe from Unfiltered, click here.

[chris_lewis]

I wrestle with that kind of proliferation as well, but I'm leery of rolling such verification into an extractor because of the double hit - which of course is already incurred b/c of the current if-guard.

@laufer, if you're trying to show a use case for if guards, that's one thing. However, don't neglect to drill in the fact that, in the case of matches (valid user), the verification will be executed twice. Minor detail for a static in-memory example, but potentially a serious bottleneck for a service backed by something like a datastore.

-chris

[soft_props]

I just wanted to point this out [1] as being pretty neat

I'd never thought of folding plans into the server. Very cool @laufer

[1]: https://github.com/webservices-cs-luc-edu/unfiltered-example-bookmarks/blob/master/src/main/scala/edu/luc/cs/webservices/unfiltered/bookmarks/Main.scala#L17

-Doug Tangren
http://lessis.me

On Fri, Mar 18, 2011 at 10:39 AM, chris_lewis [via Databinder] <[hidden email]> wrote:

I wrestle with that kind of proliferation as well, but I'm leery of rolling such verification into an extractor because of the double hit - which of course is already incurred b/c of the current if-guard.

@laufer, if you're trying to show a use case for if guards, that's one thing. However, don't neglect to drill in the fact that, in the case of matches (valid user), the verification will be executed twice. Minor detail for a static in-memory example, but potentially a serious bottleneck for a service backed by something like a datastore.

-chris

---

If you reply to this email, your message will be added to the discussion below:

http://databinder.3617998.n2.nabble.com/Simple-CRUD-example-for-comparison-with-other-frameworks-and-toolkits-tp6159509p6184752.html

To start a new topic under Unfiltered, email [hidden email]
To unsubscribe from Unfiltered, click here.

[laufer]

@Doug, I'm glad you liked that!

@Chris, excellent point! Do you agree that my BookmarksRepository trait needs to be refactored to a Session Façade/Remote Façade that provides, for each HTTP request, a single coarse-grained method including authorization?

soft_props wrote

I just wanted to point this out [1] as being pretty neat

I'd never thought of folding plans into the server. Very cool @laufer

[1]:
https://github.com/webservices-cs-luc-edu/unfiltered-example-bookmarks/blob/master/src/main/scala/edu/luc/cs/webservices/unfiltered/bookmarks/Main.scala#L17

-Doug Tangren
http://lessis.me


On Fri, Mar 18, 2011 at 10:39 AM, chris_lewis [via Databinder] <
[e-mail]> wrote:

> I wrestle with that kind of proliferation as well, but I'm leery of rolling
> such verification into an extractor because of the double hit - which of
> course is already incurred b/c of the current if-guard.
>
> @laufer, if you're trying to show a use case for if guards, that's one
> thing. However, don't neglect to drill in the fact that, in the case of
> matches (valid user), the verification will be executed twice. Minor detail
> for a static in-memory example, but potentially a serious bottleneck for a
> service backed by something like a datastore.
>
> -chris

[chris_lewis]

At least that solution would avoid the performance issue. With something like a façade to something like a session-backed cache, the hit would be less severe, but personally I would choose to avoid it all together.  

As for a "good" solution: an explicit service call works, be it an explicit dependency or something mixed in from a trait. However I feel like there is a better solution. The nice thing about a typical webapp is that authorization is safely tucked out of sight, and its results are persistent (ie a User object is on the session). I say "nice" because one needn't fiddle about with repetitious service calls.

There have been other chats about user authentication in unfiltered, and it seems the general consensus is to have a dedicated authorization trait that runs early. This frees you from validation later, but is of no help if you need a User instance. So I have two suggestions:

1) A session extractor. Auth traits could use the session and extractor without the worry of a double service hit.
2) A kind of request attribute extension to unfiltered, which would probably be analogous to servlet request attributes. Such attributes would be available for the duration of the request, and be available for access/addition to intents. I've not gotten around to exploring this extension yet, but I intend to.

These ideas need to cook a bit more. In both cases you'd have to deal with the typing issue in the extractor, and probably other things I've not thought about...

Sorry for rambling! Thanks again for starting this conversation.

-chris

[Dustin Whitney]

I've been thinking #2 would be useful too - perhaps a STM wrapped map. 

D

On Sun, Mar 20, 2011 at 12:36 PM, chris_lewis [via Databinder] <[hidden email]> wrote:

At least that solution would avoid the performance issue. With something like a façade to something like a session-backed cache, the hit would be less severe, but personally I would choose to avoid it all together.  

As for a "good" solution: an explicit service call works, be it an explicit dependency or something mixed in from a trait. However I feel like there is a better solution. The nice thing about a typical webapp is that authorization is safely tucked out of sight, and its results are persistent (ie a User object is on the session). I say "nice" because one needn't fiddle about with repetitious service calls.

There have been other chats about user authentication in unfiltered, and it seems the general consensus is to have a dedicated authorization trait that runs early. This frees you from validation later, but is of no help if you need a User instance. So I have two suggestions:

1) A session extractor. Auth traits could use the session and extractor without the worry of a double service hit.
2) A kind of request attribute extension to unfiltered, which would probably be analogous to servlet request attributes. Such attributes would be available for the duration of the request, and be available for access/addition to intents. I've not gotten around to exploring this extension yet, but I intend to.

These ideas need to cook a bit more. In both cases you'd have to deal with the typing issue in the extractor, and probably other things I've not thought about...

Sorry for rambling! Thanks again for starting this conversation.

-chris

---

If you reply to this email, your message will be added to the discussion below:

http://databinder.3617998.n2.nabble.com/Simple-CRUD-example-for-comparison-with-other-frameworks-and-toolkits-tp6159509p6189368.html

To start a new topic under Unfiltered, email [hidden email]
To unsubscribe from Unfiltered, click here.

[n8han]

As far as I can tell, the way that the bookmarks app does authentication
in most or all cases does not evaluate the verified match expression
more than once. Essentially:

unfiltered.filter. Planify {
   case req @ GET(Path(Seg("secured" :: rest)) => {
       req match {
          case Verified(user) => [response with content]
          case _ => NotAuthorized
       }
   }
}

It is only the outer partial function that is tested with isDefinedAt
before applying it (so that the filter can Pass, if it doesn't match).
If the request does match and is under "secured", at that point it is
applied to the outer PF and then the inside match is applied without any
explicit call to isDefinedAt. The `Verified.unapply` method is called
just one time, either returning Some(user) or None if no valid user
could be extracted from the request.

I'm not saying that this is the best way to do authentication with
Unfiltered. I haven't made up my mind either! But this way does not have
the inefficiency of double evaluation that you do have when the
authorization extraction is done in the "intent" partial function. (And
the same goes for the try { happy path } method.)

On the other hand, authorization is traditionally done in a centralized
place, so this could be more confusing to work with. I don't know
whether this way of doing it would scale well to larger applications or
if it would end up being a big hassle.

Nathan

[soft_props]

re #2 

The idea of request map you put things into and take out of is out of the scope of the core since we now support both Netty which isn't based on servlets as well as a traditional j2ee impl, servlet filters.  This was a bit problematic when trying to implement a transparent means passing along the identity of the oauthorized user to the next `upstream` plan in our oauth impl [1]. Any time I reach into `underlying` it ties me to that underlying impl. 

I don't think that's a solved problem yet in unfiltered so I'd be open to hearing suggestions.

[1]: http://sourced.implicit.ly/net.databinder/unfiltered-oauth/0.3.2/oauth.scala.html#9137

[chris_lewis]

I may be missing something obvious - it's early and the weekend was long - but I don't see why that matters. The half-baked idea I had is essentially a transient Map[String, Any] on RequestBinding. It doesn't matter what the underlying server framework is, only that a request now carries with it this map.