Wednesday 27 January 2010

MVC on the Google App Engine with Spring 3.0

On the Google App Engine (GAE) I can deploy and test applications very quickly. What I need now is a programming "toolkit" wich is essential and efficient.
I thought about using Grails but there is a lot of dependencies I don't want.
An implementaion based solely on the Servlet API wolud be essential but time consuming.
I decide to use the last release of the Springframework to see if it is possible to set up a project template for the GAE.

The Application Template

At this time I need two modules:
  • home with the pages "home", "about" and "description".
  • stage with some different controllers service each a page called "list" and a page called "details".
Home
This section is display-only and supports the GET method only. "Welcome" contains the list of the RSS feeds into the HEADER, but if something goes wrong the feed are simply missing. No error message is displayed.
Stage
The stage is a restricted area containing lists and forms.
This section implements the feed registration where "list" contains a short summary of all the available feeds. From this page I may add or delete a feed.
The "details" page is a form containing all the fields. The feed may be modified or deleted.

The Project Layout

The different parts of the application have to be easily identified on the filesystem. By convention the path of the java packages is build on a prefix (application.web) and using the name of the section:
application.web.home 
application.web.stage
Each section has at least one controller. The name is composed by the section name and the word controller.
application.web.home.HomeController 
application.web.stage.StageController
The stage sections contains more controllers one for each editor with restricted access. At this time I have the only the "feed" editor.
application.web.stage.FeedController
I use a similar convention for the views (JSP pages). The views are placed under WEB-INF/views:
WEB-INF/views/home/about.jsp 
WEB-INF/views/home/description.jsp 
WEB-INF/views/home/home.jsp 
 
WEB-INF/views/stage/stage.jsp 
WEB-INF/views/stage/feed.jsp 
WEB-INF/views/stage/feedDetails.jsp
The whole layout was created using Maven ( mvn archetype:generate ). I did merge "maven-archetype-quickstart" and "maven-archetype-webapp".

The Controllers

The home controller is quite simple:
package ch.clx.application.web.home;
...
@Controller
@RequestMapping("/home")
public class HomeController {
    
  @RequestMapping(method = RequestMethod.GET)
  public String get(final ModelMap model) {
    ...
    model.addAttribute("version", Version.version);
    return "home";
  }
    
  @RequestMapping(value="/about", method = RequestMethod.GET)
  public String about(final ModelMap model) {
    model.addAttribute("version", Version.version);
    return "about";
  }
    
  @RequestMapping(value="/description", method = RequestMethod.GET)
  public String description(final ModelMap model) {
    model.addAttribute("version", Version.version);
    return "description";
  }
}
The feed controller is a simple CRUD editor. The GET method returns the list of the feeds (the feed view).
  • Add (C) creates a new entry and displays the form with all the fields (detail view).
  • Delete (D) removes an existing entry.
  • Edit (R) displays an existing entry (detail view).
  • Store (U) updates an existing entry.
package ch.clx.application.web.stage;  
...  
@Controller  
@RequestMapping("/stage/feed")  
public class FeedController {  
      
  @RequestMapping(method = RequestMethod.GET)  
  public String get(final ModelMap model) {  
    ...  
    model.addAttribute("version", Version.version);  
    return "feed";  
  }  
      
  @RequestMapping(value = "/add", method = RequestMethod.POST)  
  public String add(  
    @RequestParam("id") final String id,   
    final ModelMap model) {  
    ...  
    model.addAttribute("version", Version.version);  
    return "feedDetails";  
  }  
      
  @RequestMapping(value = "/delete", method = RequestMethod.POST)  
  public String add(  
    @RequestParam("id") final String id,   
    final ModelMap model) {  
    ...  
    model.addAttribute("version", Version.version);  
    return "feed";  
  }  
      
  @RequestMapping(value="/edit", method = RequestMethod.GET)  
  public String edit(  
    @RequestParam("id") final String id,   
    final ModelMap model) {  
    ... 
    model.addAttribute("version", Version.version);  
    return "feedDetails";  
  }  
      
  @RequestMapping(value="/store", method = RequestMethod.GET)  
  public String store(  
    @RequestParam("id") final String id, 
    ..., 
    final ModelMap model) {  
    ... 
    model.addAttribute("version", Version.version);  
    return "feed";  
  }  
}

The Context File

The context file is very simple because components and URL routing are automatically discovered through annotations.

<?xml version="1.0" encoding="UTF-8"?> 
<beans 
  xmlns="http://www.springframework.org/schema/beans" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xmlns:p="http://www.springframework.org/schema/p" 
  xmlns:context="http://www.springframework.org/schema/context" 
  xsi:schemaLocation=" 
    http://www.springframework.org/schema/beans  
    http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
    http://www.springframework.org/schema/context 
    http://www.springframework.org/schema/context/spring-context-3.0.xsd" 
> 
<bean 
  class="org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping"/> 
<bean 
  class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter"/> 
 
<context:component-scan base-package="gae.application"/>

<bean 
  id="viewResolver" 
  class="org.springframework.web.servlet.view.ResourceBundleViewResolver"/>

</beans>

The Deployment Descriptor

web.xml
<?xml version="1.0" encoding="UTF-8"?> 
<web-app 
  id=2gae-application" 
  version="2.4" 
  xmlns="http://java.sun.com/xml/ns/j2ee" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee  
      http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"/> 
  <display-name>gae-application</display-name> 
  <servlet> 
    <display-name>Dispatcher Servlet</display-name> 
    <servlet-name>Dispatcher-Servlet</servlet-name> 
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> 
    <init-param>
      <param-name>contextConfigLocation</param-name>
      <param-value>/WEB-INF/WebCtx.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping
    <servlet-name>Dispatcher-Servlet</servlet-name>
    <url-pattern>/bo/*</url-pattern>
  </servlet-mapping>
  <welcome-file-list>
    <welcome-file>index.jsp</welcome-file>
  </welcome-file-list>
</web-app>

The GAE Deployment Descriptor

appengine-web.xml
<?xml version="1.0" encoding="UTF-8"?> 
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0"> 
<application>gae-application</application> 
<version>beta-1</version> 
<system-properties> 
<property 
  name="java.util.logging.config.file" 
  value="WEB-INF/classes/logging.properties"/>
</system-properties> 
</appengine-web-app>

The Maven Archetype

At the end of my experiment I pack my template application in a Maven archetype. I is enough to add the following pom.xml file to the project:
<?xml version="1.0" encoding="UTF-8"?>
<project
  xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
    http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>cloud.gae.application</groupId>
  <artifactId>gae-application</artifactId>
  <version>1.0</version>
  <packaging>war</packaging>
  ...
  <build>
    <finalName>${artifactId}</finalName>
    ...
  </build>
</project>
Now type mvn archetype:create-from-project to generate the archetype. Change the directory ( cd target/generated-sources/archetype ) and execute mvn install .
Using mvn archetype:generate now I may generate a GAE application ready to use. It does nothing but as starter kit for GAE applications is not bad.

Saturday 16 January 2010

URL Shortening Service

A URL shortening service is not essential for the most part of the applications in the WEB and may be dangerous because the surfer doesn't see what lies behind the short URL. But if I would like to publish something on a short messaging system like Twitter I need a very short URL.

The solution would be to register a very short domain like goo.gl and to run my own shortener, but for my application the additional effort is out of scope. So I need to connect to an existing URL shortening service.

The beginner has a first obstacle: where to find an inexpensive reliable URL shortening service. Using a search engine I did found a lot of service which are also free of charge, but no way to proof if the are also reliable.

Service Requirements

Since I want to generate the short URL into the background I need a service with API support. Additionally stating from the short URL I want to obtain the original (long) URL.

The candidates

I didn't want to check all the services, for instance four of them are enough.

bit.ly

The service is free of charge, but  requires a registration and checks each request using an API key.

  • Homepage with a clear description of the service.
  • Simple well documented API.
  • Shorten and expand functions.
  • JSON and XML formatted response.
  • Simple but nice statistics summary.
  • Blog

I like bit.ly and is currently my first choice.

is.gd

The service is free of charge and doesn't require a registration.

  • Homepage with a clear description of the service.
  • Very simple API with documentation.
  • Shorten only function.
  • Plain text (URL only) response.

This service is very spartan; I noticed that after two or three days the short URL was different. If a very short URL is required id.gd may be a good choice. 

snipurl.com

The service is free of charge, but requires a registration and checks each request using an API key.

  • Homepage with a clear description of the service.
  • Simple well documented API (Firefox doesn't display the page and with the MSIE there are some curious side effects, try with is-there-a-snipr-snipurl-api).
  • Shorten and expand functions.
  • XML and plain text (URL only) formatted response.
  • Simple statistics but searchable statistics.
  • Help

SnipURL exists since the year 2000, which is a sign of reliability. I miss the JSON formatted response, but the service is good.

tinyurl.com

The service is free of charge and doesn't require a registration.

  • Homepage with a clear description of the service.
  • Very simple API.
  • Shorten only function.
  • Plain text (URL only) response.

The service is very spartan, but the short URL is the same over the time.

Test the services

I did test this services for a while using a very simple HTTP client.

Client Requirements

The client have to be essential, but should be able to:

  • retry the connection three times before to give up.
  • follow the redirects (max three levels of indirection).
  • support GET & POST.
  • support the secure socket layer (SSL, nice to have).

Implementation

I use a very simple implementation done in Java and based on the HttpURLConnection, the following snippet shows the essential part of the code:

...
URL url = URL("http://myserver.com");
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod("GET");
conn.addRequestProperty("User-Agent","FastShortGood/1.0 Java/6");
...
// Extract the character set from the "Content-Type"
// header using a regular expression. If none available
// assume "UTF-8"
...
StringBuilder buf = new StringBuilder();
InputStream is = conn.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is, charset));
String line;
while (null != (line = br.readLine())) {
  buf.append(line).append("\n");
}
String body = buf.toString();

// Extract the short URL from response body using a
// regular expression.

For the complete implementation I need about 300 lines of code. The most of them handles exceptions and HTTP respose codes. The lightweight client works well with all of the shortener services.

Sunday 3 January 2010

Receiving Emails with the Google App Engine

Last year, in November, I did explore for the first time the Google App Engine for Java (GAE/J). One of the feature my application provides is to accept an email and translate it to an RSS 2.0 item.
The first version of the receiver was done using JavaMail. I did experiment some troubles with the "Quoted-Transfer-Encoding", sometimes the content was broken.
Using the current version of the GAE/J (1.3.0) the emails content received from Gmail or from Thunderbird is correct and the channel seems to be reliable.

Mime4J

An alternative for receiving email is the James Mime4J project.
The library doesn't have dependencies and works on the GAE/J. It is quite simple to use; instead of:

import java.io.InputStream;
import java.util.Properties;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
...
InputStream input = request.getInputStream(); // ServletRequest
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage message = new MimeMessage(session, input); // MessagingException
...
  
you have to use:
import java.io.InputStream;
import org.apache.james.mime4j.message.Message;
import org.apache.james.mime4j.message.MessageBuilder;
import org.apache.james.mime4j.parser.ContentHandler;
import org.apache.james.mime4j.parser.MimeStreamParser;
import org.apache.james.mime4j.storage.MemoryStorageProvider;
...
InputStream input = request.getInputStream(); // ServletRequest
Message message = new Message();
ContentHandler handler = new MessageBuilder(message, new MemoryStorageProvider());
MimeStreamParser parser = new MimeStreamParser();
parser.setContentHandler(handler);
parser.parse(input); // MimeException
...
The result seems to be better than the one which uses JavaMail. The mails received from the MS Outlook are readable. It remains a small problem with the encoding which I will investigate later this year.

Incoming Mail Syntax

The syntax of the incoming mail was changed:
description
line 1
line 2
...
line n
end

link
http://myserver/mylink
end

categories
category 1
category 2
...
category n
end
Each command block starts with a keyword and ends with end

Informations

The email address of the Mime4J service is mime4j@example-rest.appspotmail.com, the JavaMail based receiver still works with rss2@example-rest.appspotmail.com
The application URL is example-rest.appspot.com.