Commands
Scalatra includes a very sophisticated set of validation commands.
These allow you to parse incoming data, instantiate command objects, and automatically apply validations to the objects. This sounds like it might be quite complex, but once you’ve got the (quite minimal) infrastructure in place, it can dramatically simplify your code.
Let’s say we’ve got a Todolist application, and it contains a simple Todo class which is used for persistence:
// A Todo object to use as a data model
case class Todo(id: Integer, name: String, done: Boolean = false)
Using a command, a controller action for creating and saving a new Todo object might look like this:
post("/todos") {
(command[CreateTodoCommand] >> (TodoData.create(_))).fold(
errors => halt(400, errors),
todo => redirect("/")
)
}
You define the command separately, tell it which case class type it operates upon, and set up validations inside the command:
object CreateTodoCommand {
// Putting the implicit conversion in the companion object of the create todos command ensures it's the default fallback
// for implicit resolution.
implicit def createTodoCommandAsTodo(cmd: CreateTodoCommand): Todo = Todo(~cmd.name.value)
}
class CreateTodoCommand extends TodosCommand[Todo] {
val name: Field[String] = asType[String]("name").notBlank.minLength(3)
}
Several things happen when execute
(>>
) is called. First, validations
are run, and then the command is either converted implicitly to the parameter the
function accepts or just passed in as a command into that function.
The result of that function is a ModelValidation.
The CreateTodoCommand
can automatically read incoming POST params or JSON,
populate itself with whatever info it read, run validations to ensure that
the name
property is a non-empty String
with at least 3 characters,
and then, in this case, save the Todo object.
Note that in this code, the incoming params are not automatically pushed onto a new instance of the Todo case class. This is because Scalatra users habitually use wildly varying approaches to persistence frameworks and have very different needs when it comes to data validation.
What the CreateTodoCommand
object gives you instead, is a way to componentize
and re-use the same Command object across any part of your application which
requires the creation of a Todo, and easily apply validation conditions
based on incoming parameters.
Since validation commands in Scalatra have nothing to do with your
chosen persistence library, the concepts of commands and validation are
completely de-coupled from the concept of persistence. You might want to have
the execute
method of a command trigger a persistence function; just as easily,
you could serialize the Todo object and send it off to a queue, attach it to
another object, or transform it in some way.
This has some benefits:
- data validation and persistence are de-coupled.
- the validations DSL makes setting validation conditions very easy.
- validations are taken care of right at the front door of your application. Bad data never gets deep into your stack.
- error handling and validation failures are more convenient, and you can use Scala’s pattern matching to determine appropriate responses.
The TodoList application
To see how Scalatra’s commands work, let’s create a TodoList application. It’ll allow you to use Scalatra’s command support to validate incoming data and do data-related work.
Downloading the sample project
This tutorial will start by generating a fresh project, talk you through the project setup, and then show you several different ways of using commands in Scalatra.
Generating the project
Generate a new project. We’ll use org.scalatra.example.commands
domain as
a namespace, you can change to your own domain throughout the codebase.
$ sbt new scalatra/scalatra-sbt.g8
organization [com.example]: org.scalatra
package [com.example.myapp]: org.scalatra.example.commands
name [My Scalatra Web App]: TodoList
servlet_name [MyServlet]: TodosController
scala_version [2.9.2]:
version [0.1.0-SNAPSHOT]:
Template applied in ./todolist
Start the application:
$ cd todolist
$ sbt
Now you’re in the sbt shell, start the server and enable recompilation:
container:start
~;copy-resources;aux-compile
Setting up a model and fake datastore
Before we start actually building the controller, let’s set up some fake data.
Add two folders in org.scalatra.example.commands
: call one models
and the other data
.
Inside the models
folder, add a file called Models.scala
, with the following contents:
package org.scalatra.example.commands.models
// A Todo object to use as a data model
case class Todo(id: Integer, name: String, done: Boolean = false)
You might drop a few more models in there later on, but for now we’ll just define a single Todo
model.
Next, inside the data
folder, add a file called TodoData.scala
, with the following contents:
package org.scalatra.example.commands.data
import org.scalatra.example.commands.models._
import java.util.concurrent.atomic.AtomicInteger
object TodoData {
/** A counter variable to fake out auto-incrementing keys for us **/
val idCounter = new AtomicInteger(3)
/**
* Some fake data so we can simulate retrievals.
*/
var all = List(
Todo(1, "Shampoo the cat"),
Todo(2, "Wax the floor"),
Todo(3, "Scrub the rug"))
/** Returns the number of Todos which are not yet complete. **/
def remaining = {
all.filterNot(_.done == true).length
}
/** Adds a new Todo object to the existing list of todos, then sorts the list.
*/
def add(todo: Todo): List[Todo] = {
all ::= todo
all = all.sortBy(_.id)
all
}
/** Instantiates a new `Todo` object with an auto-incremented primary key id. **/
def newTodo(name: String) = Todo(idCounter.incrementAndGet, name)
}
For the purposes of this example, we won’t bother with persisting our Todos
to disk. The TodoData
object acts as temporary storage of our Todos, and
has methods on it to access all Todos, find out how many haven’t yet been
completed (using the remaining
method), and instantiate a new Todo
object with an auto-incrementing integer primary key.
Retrieving objects in a controller
Let’s move back to the TodosController, and get the commands working.
Just to see if everything is working, let’s try and retrieve a single Todo.
First, import the definitions for the classes we just added:
import models._
import data._
Then put this action in your controller
get("/todos/:id") {
TodoData.all find (_.id == params("id").toInt) match {
case Some(todo) => todo
case None => halt(404)
}
}
Hitting http://localhost:8080/todos/1 should now show you a Scala representation of the first Todo object:
Todo(1,Shampoo the cat)
All pretty simple so far. Let’s drop in some code which will allow us to add a new Todo object, using a Scalatra command.
Commands in Scalatra
Scalatra’s commands are built using the classical
Gang of Four (Gof)
Command pattern, with some
small variations. In its simplest form, a command object has one method,
execute
, which calls a method on another class, the receiver. The
command object can carry data with it, be passed from method to method, and
finally tell the receiver to do some work when the execute
method is called.
It’s a way to increase flexibility and de-couple calling methods from
receivers.
In Scalatra, Command
objects have a few things
added to them which aren’t in the traditional GoF Command Pattern.
First, they’re able to automatically read incoming parameters
and populate themselves with data. Second, they can also run validations on the
parameters to ensure data correctness.
Adding a command to persist Todo objects
We’ll need a file in which to place our commands. Make a
new folder in org.scalatra.example.commands
, and call it commandsupport
.
Then create a new file in that folder, calling it TodoCommands.scala
. This
will store all of the Todo-related commands so they’re in one place.
To start with, you’ll need to add command support to your application.
"org.scalatra" %% "scalatra-commands" % "2.3.2",
TodoCommands.scala
can look like this:
package org.scalatra.example.commands.commandsupport
// the model code from this application
import org.scalatra.example.commands.models._
// the Scalatra commands handlers
import org.scalatra.commands._
abstract class TodosCommand[S] extends ParamsOnlyCommand
object CreateTodoCommand {
// Putting the implicit conversion in the companion object of the create todos command ensures it's the default fallback
// for implicit resolution.
implicit def createTodoCommandAsTodo(cmd: CreateTodoCommand): Todo = Todo(~cmd.name.value)
}
/** A command to validate and create Todo objects. */
class CreateTodoCommand extends TodosCommand[Todo] {
val name: Field[String] = asType[String]("name").notBlank.minLength(3)
}
There are a few things going on here, so let’s break it apart. First, there are some imports: the model code for our application, and the command support.
The next thing is the abstract class TodosCommand
. This sets up an
abstract base class for all of our commands to inherit from, so we don’t
need to keep on repeating the extends ModelCommand[T]
in every
command we make. It inherits from two other classes, both of which are
built into Scalatra: ModelCommand[S]
and ParamsOnlyCommand
.
ModelCommand[S]
is a very small subclass of Scalatra’s base
Command
object. It’s just a Command which takes a single type parameter,
and it’s abstract. It gives the Command object the ability to know which
case class type it’s operating upon.
ParamsOnlyCommand
is basically a Command with type conversion enabled.
It allows population of a command’s fields from incoming params.
Finally, there’s the concrete CreateTodoCommand
class. This is the first
command object we’ll use, and its job will be to validate incoming params
for a Todo object. Once that’s done, we’ll use the command receiver’s handle
method to persist a new Todo object in our fake datastore.
Validations
CreateTodoCommand has an interesting val
hanging around in the class
body:
val name: Field[String] = asType[String]("name").notBlank.minLength(3)
This indicates to the command that a Todo has a field called name
,
which needs to be a String
. There are two validations: the name must
be notBlank
(i.e. it can’t be an empty string or a null value), and
it must have a minLength(3)
(i.e. it must have a minimum length of
3 characters).
A full list of available validations is available in the Validators API docs.
That’s it for the command setup. Now that we’ve got a command which can create Todos, let’s use it in a controller action to create a Todo object.
Using the new command in a controller action
Back in TodosController, let’s add a new route, and set it up to use this new capability.
post("/todos") {
val todo = new Todo(-1, params("name"))
TodoData.add(TodoData.newTodo(params("name")))
redirect("/")
}
This works fine, but if there are a lot incoming parameters, it can get a
little tiresome extracting them from the params
bag and populating the
object. Commands give us a better way, with the bonuses of convenient
validation and error handling.
Before we can use any command-related code, we’ll need to import it into our controller class. You’ll need:
// the Scalatra command handlers
import org.scalatra.commands._
// our own Command classes
import commands._
Fill in the action for post("/todos")
like this:
post("/todos") {
(command[CreateTodoCommand] >> (TodoData.add(_))).fold(
errors => halt(400, errors),
todo => redirect("/")
)
}
This won’t compile yet. Before we make it compile, let’s take a line by line look at the action, to understand what it’s doing.
First, we instantiate a new CreateTodoCommand:
val cmd = command[CreateTodoCommand]
This gives us a new CreateTodoCommand, which knows it’s operating on the Todo model, and can ingest incoming params to automatically populate the model’s fields.
We then tell TodoData, our fake datastore, to execute
the cmd
. At
present, this is holding up compilation: TodoData
has no
execute
method. Let’s fix this.
First, let’s make a logger. This isn’t strictly necessary, but it’s a nice thing to have around.
Create a new folder, utils
, in org.scalatra.example.commands
, and
put the following code into Logger.scala
inside it:
package org.scalatra.example.commands
package utils
import grizzled.slf4j.Logger
trait Logging {
@transient lazy val logger: Logger = Logger(getClass)
}
This gives us a logger object.
Open up data/TodoData.scala
, and add the following code.
At the top of the file, add:
// the Scalatra command handlers
import org.scalatra.commands._
// our commands
import org.scalatra.example.commands.commandsupport._
// our logger object
import org.scalatra.example.commands.utils.Logging
Now to get things compiling again. Add these imports:
import scala.util.control.Exception._
import org.scalatra.validation._
These imports give you access to exception handling and validation code, which we’ll use in a moment.
In Scalatra, when you call execute
on your Command, you’re telling it to
do two things:
- run validations on all fields defined in your command
- it passes itself into the function passed to execute
One more thing and it’ll compile. Change the add
method so that it returns a
ModelValidation[Todo]
, and add some error handling:
def add(todo: Todo): ModelValidation[Todo] = {
allCatch.withApply(errorFail) {
all ::= todo
all = all.sort((e1, e2) => (e1.id < e2.id))
todo.successNel
}
}
/** Throw a validation error if something explodes when adding a Todo **/
def errorFail(ex: Throwable) = ValidationError(ex.getMessage, UnknownError).failNel
Your code should now compile.
Let’s go through that last piece of the puzzle. The heart of the add
method still does the same thing: it adds a new Todo
object to the list
of all todos, and sorts the list.
The add
method returns a ModelValidation[Todo], which is carried around in the
todo.successNel. Think of successNel
as being like a two part variable
name. The result is either Success[Model]
or
Failure[NonEmptyList[ValidationError]]
. So you’re getting
back either “success” OR a non-empty list (“Nel”). This type signature is
in turn dictated by the return value needed by the handle
method, above.
If any exceptions happen as we’re doing work here, the errorFail
method
will be called, due to the allCatch.withApply
(which is equivalent to a
try {} catch {}
block.
You should now be able to add a new Todo
object to the datastore.
Let’s quickly add a method to see the ones we’ve currently got:
get("/todos") {
TodoData.all
}
Your list of Todos should look like this:
List(Todo(1,Shampoo the cat,false), Todo(2,Wax the floor,false), Todo(3,Scrub the rug,false))
Try adding one:
curl -X post -d name="Do that thing" http://localhost:8080/todos
Hit refresh and you should see the new todo in the list.
Let’s recap on what’s happened here.
First, the incoming params (in this case only name
) hit the post("/todos")
method. A new CreateTodoCommand
is instantiated:
val cmd = command[CreateTodoCommand]
Note: The method command[Foo]
comes courtesy of Scalatra’s command support.
Next, the command gets executed: TodoData.execute(cmd)
.
Calling execute
on the command causes all validations to run, and then
the handle
method is called. Note: validations could fail!
In this case, the handle
command as implemented in TodoData
adds a
new Todo object to the list of todos:
add(newTodo(c.name.value getOrElse ""))
The add
method attempts to add the new Todo object to the datastore.
This could also potentially fail.
What happens in the failure cases? This is determined by the remainder
of the TodoData.execute
method call:
(cmd >> TodoData.add(_)).fold(
errors => halt(400, errors),
todo => redirect("/")
)
If we get back errors (from either the validations or the allCatch
block),
we halt with a 400 status. If we get back a todo
, we redirect to “/”.
At this point, your project should be very similar to what’s tagged in the example project’s Git repository, in the params binding example.
Using Scalatra’s commands with JSON
So far, we’ve been doing everything with params data only. We can easily switch to using JSON instead. Conveniently, when you enable the JSON support with the commands, you can use either regular POST params, e.g. from a web form, OR JSON documents, transparently and with no extra effort.
Here’s how it works.
Add the following to project/build.scala:
"org.json4s" %% "json4s-jackson" % "3.2.11",
"org.scalatra" %% "scalatra-json" % "2.3.2",
This adds dependencies on Scalatra’s JSON-handling libraries.
Next, add the following imports to TodosController.scala
, so that the
controller has access to the new JSON libraries:
// Json handling
import json._
import org.json4s.{DefaultFormats, Formats}
Next, add with JacksonJsonParsing with JacksonJsonSupport
to the controller
instead of ParamsOnlySupport
. This will give your controller the ability
to automatically parse incoming params using the Jackson JSON-handling library.
The last thing we’ll need to do in the controller is to add json format support by putting the following code in the class body:
// add json format handling so the command can do automatic conversions.
protected implicit val jsonFormats = DefaultFormats
If you want to, you can set the default format of all actions in your controller to be JSON, by adding this to the body of the TodosController class:
before() {
contentType = formats("json")
}
and adding with JValueResult
to the TodosController class declaration.
That’s it for your controller. Now let’s fix up the commands.
In commands/TodoCommands.scala
, remove with ParamsOnlySupport
from the
abstract class TodosCommand[S]
and add with JsonCommand
instead.
Add the following imports to the top of the file:
// Json handling
import json._
import org.json4s.{DefaultFormats, Formats}
And again, we’ll need to give the class the ability to do automatic format conversions to and from JSON, so put the following code into the body of the CreateTodoCommand class:
// add json format handling so the command can do automatic conversions.
protected implicit val jsonFormats = DefaultFormats
Take a look at the output of http://localhost:8081/todos
It should now have changed to be JSON:
[
{"id":1,"name":"Shampoo the cat","done":false},
{"id":2,"name":"Wax the floor","done":false},
{"id":3,"name":"Scrub the rug","done":false}
]
We can still add a new Todo object using a regular POST:
curl -X post -d name="Do that thing" http://localhost:8080/todos
[
{"id":1,"name":"Shampoo the cat","done":false},
{"id":2,"name":"Wax the floor","done":false},
{"id":3,"name":"Scrub the rug","done":false},
{"id":4,"name":"Do that thing","done":false}
]
We’ve also got a new capability: we can POST a JSON document to
http://localhost:8080/todos, and the CreateTodoCommand
will handle that
as well:
curl -X post -i -H "Content-Type: Application/JSON" -d '{"name":"Find out how to use JSON commands", "done":true }' http://localhost:8080/todos
Scalatra reads the Content-Type
header, takes the hint that what’s coming in the
door is JSON, and informs the CreateTodoCommand
of that.
Alternately, if you prefer, you can just as easily send a format
parameter
instead of a Content-Type
header:
curl -X post -i -d '{"name":"Find out how to use JSON commands", "done":true }' http://localhost:8080/todos?format=json
At this point, your project should be very similar to the JSON commands Git example. Take a look at that code if you’re having any problems.
Writing your own validations.
Scalatra gives you a fairly comprehensive list of pre-built validations, but you can also write your own custom validators fairly easily.
A Scalatra Command is partly composed of Field objects, each of which has a FieldDescriptor which acts as a kind of builder for the Field.
In order to write a validator, we need to do two things.
First, we need to write a class to carry around our custom validations.
Second, we implicitly extend Scalatra’s FieldDescriptor class so that it’s got access to our new validation. Let’s see this in action.
We’ll need to decide what kind of validation to make. Since all-lower-case
text is the mark of the pathologically lazy, let’s hold ourselves to a
higher standard, and define a validation which will force users of our
application to capitalize the first letter of the name
field in their
Todo
objects.
Open up your TodoCommands.scala
file, and drop this into it above the
abstract class TodosCommand
:
/**
* A class to keep our custom String validations in.
*
* Note that it takes a FieldDescriptor[String] binding as a parameter.
* This is so that we can extend the FieldDescriptor.
*/
class TodosStringValidations(b: FieldDescriptor[String]) {
// define a validation which we can apply to a [Field]
def startsWithCap(message: String = "%s must start with a capital letter.") = b.validateWith(_ =>
_ flatMap { new PredicateValidator[String](b.name, """^[A-Z,0-9]""".r.findFirstIn(_).isDefined, message).validate(_) }
)
}
The TodosStringValidations
class is just a container for our
validations.
Inside it, there’s a startsWithCap
function, which takes a String
parameter for the validation message
, and can apply itself to a
FieldDescriptor[String]
via binding b
, using b.validateWith
.
The heart of the validation function is this bit of code here:
new PredicateValidator[String](b.name, """^[A-Z,0-9]""".r.findFirstIn(_).isDefined
To paraphrase, we’re going to run a validation
on the FieldValidator’s name, and that the validation should pass if
the regex [A-Z,0-9]
matches the first letter of the incoming string.
What’s that _ flatMap
doing there at the start of the validation
function? This takes a bit of explaining. Eventually, we’re going to chain together our new startsWithCap
validation with the rest of the
validations we’ve already defined, like this:
val name: Field[String] = asType[String]("name").notBlank.minLength(3).startsWithCap()
Validations are evaluated in a chain, starting on the left, and proceeding rightwards. Each validation condition is a logical AND.
Let’s assume that we try to validate a new Todo
with the name
“Walk the dog”.
A successful validation for a name
of Walk the dog
is of type
Success("Walk the dog")
. In contrast, a failed validation returns a
Failure(ValidationError)
with a failure message inside it, and no more
validations in the chain are run.
When our custom validation runs, it is taking as input the output of the
previous validation function. So in our case, the success output of
.minLength(3)
is fed into _
and forms the input for our startsWithCap
function.
The use of flatMap
in that function is a
Scala trick
to pull the value "Walk the dog"
out of Success("Walk the dog")
, because a Validation’s return type operates much like an Either
from the stdlib - it can be considered a 2-value sequence, with a type
signature something like this:
Validation[Failure(error), Success(data)]
Back to the task at hand.
What we need to do now is make our application aware of our new validation code, and then apply it.
Scalatra’s FieldDescriptor
trait already exists, but we can use the
extension method technique to add in our new validation code.
Let’s add to our abstract TodosCommand
class:
abstract class TodosCommand[S] extends JsonCommand {
/**
* Extending the [org.scalatra.commands.FieldDescriptor] class with our [TodosStringValidations]
*
* This adds the validation to the binding for the FieldDescriptor's b.validateWith function.
*/
implicit def todoStringValidators(b: FieldDescriptor[String]) = new TodosStringValidations(b)
}
Using implicit def
, we’re decorating Scalatra’s FieldDescriptor[String]
with our new TodosStringValidations(b)
. This makes the code available
for use in our application.
So let’s use it. Now that we’ve defined the validator and imported it,
all we need to do is add .startsWithCap()
to our validation line:
val name: Field[String] = asType[String]("name").notBlank.minLength(3).startsWithCap()
It’s worth noting that we could just as easily have defined our new validation in a library, imported it, and used it in our application. This gives you the ability to build up whatever custom validators you’d like and carry them around between projects.