Ubuntu Server as Inexpensive Platform for J2EE

Due to the very small budget of the project I had to find a good but inexpensive platform. The development server is an old single core single CPU Pentium 4 pizza-box with 200 GB hard disk. The productive environment will be much better, but OS and middle-ware should work out-of-the-box and require a very little maintenance.

Network Management on Fedora 12

Recently I had to activate a second network interface on a Fedora 12 installation at work. The server has two interfaces (eth0 and eth1) but only one (eth0) was active and allowing connections from in-house only. The project wishes to provide a preview to the customer so I had to move the server to the DMZ. To avoid collisions between my work as system administrator and the development team I decided to activate the second interface(eth1) for the DMZ.

Android 2.2 Froyo on the HTC Desire: first experiences

I did upgrade my HTC Desire from Android 2.1 to 2.2. It's possible to force an upgrade without waiting (settings -> about phone -> system software updates -> check now).

Holidays in Italy with Android

Last month I did spend some time searching the best rate for using my smartphone in Italy.

Developing with the Android Emulator: the Beginning

The documentation on Android Developers is very well done, but there is a lot of informations. Often the informations I need are spread over many different pages.

A quick setup for the development environment is easy to find. There is a lot of good sources about how to setup the virtual device, how to create a new project using the IDE and so on. What I missed is a collection of practical advices.

How To: GIT repository on Solaris 10

Last year I did replace my SVN software repository with GIT.

Last moth I began to use GIT also at the office because I will be able to roll back the experimentations sometimes I do in my projects. The history of the Eclipse IDE isn't enough to archive the goal. GIT creates locally a full featured repository with tags and branches. Any time I may switch to a different branch, so I may work, at the same time, on the mainstream and on the experimental features.

Blogging with Android: Blogaway, Wordpress

Last month I decided to buy a smartphone. I'm a UNIX developer since long time: my choice the HTC Desire.

Display

The display of the smartphone is limited in space. The layout of my blog on Blogger may be squeezed and is readable even with the zoom function.

The layout I use on Wordpress isn't readable on the smartphone, but the mobile site of Wordpress is really good.

Blogging

At the beginning everything seemed unconfortable, but once I did understand how to work, what to select and in which sequence bloggging was easy.

Blog-Away

Blogaway is a Blogspot client for Android. It's everything I need to write down my ideas.

This article was written using the smartphone and Blogaway travelling by train.

Each article may be published as draft and improved later on the PC. The text may be formatted using bold, italic or a color. It is possible to add links, pictures and videos. You may scroll the text very quickly. I like this application because it is simple and efficient.

You may find more informations about Blogaway on blogspot.com or on  Twitter.

WordPress

Read my article on Wordpress

Conclusion

Bogging with the smart-phone is possible as additional way to add articles. It is suitable for short and simple text.

If a blog is readable or not depends on the theme.

Posted via Blogaway

Small J2EE Fedora Development Server On Amazon EC2.

In the last 10 years I used a LINUX server at home to develop my private applications. Twice a year I spend a weekend to update/upgrade the server. Now I should replace the old hardware with a newest one.

Instead of buying a new server I decided to search for alternative solutions. Due to the small amount of time I really use the server, the solution have to be cheap. At the same time, due to the kind of experiments I do, I need root access to the instance.

I signed up for Amazon S3 and EC2 to be able to run a LINUX AMI. The first question is: where may I find a Fedora AMI?

The best solution for me seems to be a self defined AMI built using Elastic Server. My server is based on Fedora 10 and contains the JDK 1.6.0_13, Tomcat 6.0.18, PHP 5.2.6, MySQL 5.0.67 and Apache2. Almost the same configuration I use at home.

Elastic Server build, deploys and starts the AMI automatically. I had to define a S3 bucked, something like daniele-ami where the service will copy the image of the AMI. The builder needs to access my account at Amazon to store the image and start the AMI, so I define a dedicated Access Key. As soon as the new instance works I disable the dedicated key. The key may be re-enabled each time I want to deploy a new AMI.

Elastic Server sends an email to the account owner containing all the informations to access the the running instance.

The login (cftuser) may be performed using PuTTY (Windows) or ssh (UNIX). To gain root access I have to use the command sudo su - . A quick check of the environment confirms that everything works as expected.

Add a Data Volume

Since I don't work with a reserved instance when the instance terminates all changes made after the startup are lost. To reduce the inconvenience I define a 10 GB EBS volume using the AWS console. Note: the volume have to be defined in the same zone as the EC2 instance.

The volume is attached as device sdf (/dev/sdf)

Now the procedure is the same used to add a disk to any LINUX system:

1. Use fdisk (fdisk /dev/sdf) to delete all existing partitions (select d).
2. Save the new (empty) partition table (select w).
3. Add a new partition (select n).
4. Save the new partition table (select w).
5. Use mkfs (e.g. mkfs -t ext3 /dev/sdf1) to format the patition.
6. Use tune2fs (e.g. tune2fs -L /data /dev/sdf1) to put a label on the patition.
7. Create the mount point (e.g. mkdir /data).
8. Modify the file /etc/fstab as displayed into the next section.
9. Mount the partition (e.g. mount /data).

# /etc/fstab: static file system information.
# AMI version
#
# 'file system' 'mount point'   'type'  'options'       'dump'  'pass'
/dev/sda1       /               ext3    defaults,errors=remount-ro 1    1
/dev/sda2       /mnt            ext3    defaults        0       0
/dev/sda3       none            swap    sw              0       0
proc            /proc           proc    defaults        0       0
none            /sys            sysfs   defaults        0       0
none            /dev/shm        tmpfs   defaults        0       0
none            /dev/pts        devpts  gid=5,mode=620  0       0
LABEL=/data     /data           ext3    defaults        1       2

After a new start the last three steps of the previous list have to be done again to remount the disk. I do it using the following script:

#!/bin/bash
mkdir /data
echo "LABEL=/data     /data           ext3    defaults        1       2" >> /etc/fstab
/bin/mount /data

Execute:

• sudo su -
• vi setup
• copy & paste
  
• sh setup

The cost for an afternoon of programming and testing is $0.40 which is really cheap.

MVC on the Google App Engine with Spring 3.0

On the Google App Engine (GAE) I can deploy and test applications very quickly. What I need now is a programming "toolkit" wich is essential and efficient.
I thought about using Grails but there is a lot of dependencies I don't want.
An implementaion based solely on the Servlet API wolud be essential but time consuming.
I decide to use the last release of the Springframework to see if it is possible to set up a project template for the GAE.

The Application Template

At this time I need two modules:

• home with the pages "home", "about" and "description".
• stage with some different controllers service each a page called "list" and a page called "details".

Home
This section is display-only and supports the GET method only. "Welcome" contains the list of the RSS feeds into the HEADER, but if something goes wrong the feed are simply missing. No error message is displayed.
Stage
The stage is a restricted area containing lists and forms.
This section implements the feed registration where "list" contains a short summary of all the available feeds. From this page I may add or delete a feed.
The "details" page is a form containing all the fields. The feed may be modified or deleted.

The Project Layout

The different parts of the application have to be easily identified on the filesystem. By convention the path of the java packages is build on a prefix (application.web) and using the name of the section:

application.web.home 
application.web.stage

Each section has at least one controller. The name is composed by the section name and the word controller.

application.web.home.HomeController 
application.web.stage.StageController

The stage sections contains more controllers one for each editor with restricted access. At this time I have the only the "feed" editor.

application.web.stage.FeedController

I use a similar convention for the views (JSP pages). The views are placed under WEB-INF/views:

WEB-INF/views/home/about.jsp 
WEB-INF/views/home/description.jsp 
WEB-INF/views/home/home.jsp 
 
WEB-INF/views/stage/stage.jsp 
WEB-INF/views/stage/feed.jsp 
WEB-INF/views/stage/feedDetails.jsp

The whole layout was created using Maven ( mvn archetype:generate ). I did merge "maven-archetype-quickstart" and "maven-archetype-webapp".

The Controllers

The home controller is quite simple:

package ch.clx.application.web.home;
...
@Controller
@RequestMapping("/home")
public class HomeController {
    
  @RequestMapping(method = RequestMethod.GET)
  public String get(final ModelMap model) {
    ...
    model.addAttribute("version", Version.version);
    return "home";
  }
    
  @RequestMapping(value="/about", method = RequestMethod.GET)
  public String about(final ModelMap model) {
    model.addAttribute("version", Version.version);
    return "about";
  }
    
  @RequestMapping(value="/description", method = RequestMethod.GET)
  public String description(final ModelMap model) {
    model.addAttribute("version", Version.version);
    return "description";
  }
}

The feed controller is a simple CRUD editor. The GET method returns the list of the feeds (the feed view).

• Add (C) creates a new entry and displays the form with all the fields (detail view).
• Delete (D) removes an existing entry.
• Edit (R) displays an existing entry (detail view).
• Store (U) updates an existing entry.

package ch.clx.application.web.stage;  
...  
@Controller  
@RequestMapping("/stage/feed")  
public class FeedController {  
      
  @RequestMapping(method = RequestMethod.GET)  
  public String get(final ModelMap model) {  
    ...  
    model.addAttribute("version", Version.version);  
    return "feed";  
  }  
      
  @RequestMapping(value = "/add", method = RequestMethod.POST)  
  public String add(  
    @RequestParam("id") final String id,   
    final ModelMap model) {  
    ...  
    model.addAttribute("version", Version.version);  
    return "feedDetails";  
  }  
      
  @RequestMapping(value = "/delete", method = RequestMethod.POST)  
  public String add(  
    @RequestParam("id") final String id,   
    final ModelMap model) {  
    ...  
    model.addAttribute("version", Version.version);  
    return "feed";  
  }  
      
  @RequestMapping(value="/edit", method = RequestMethod.GET)  
  public String edit(  
    @RequestParam("id") final String id,   
    final ModelMap model) {  
    ... 
    model.addAttribute("version", Version.version);  
    return "feedDetails";  
  }  
      
  @RequestMapping(value="/store", method = RequestMethod.GET)  
  public String store(  
    @RequestParam("id") final String id, 
    ..., 
    final ModelMap model) {  
    ... 
    model.addAttribute("version", Version.version);  
    return "feed";  
  }  
}

The Context File

The context file is very simple because components and URL routing are automatically discovered through annotations.

<?xml version="1.0" encoding="UTF-8"?> 
<beans 
  xmlns="http://www.springframework.org/schema/beans" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xmlns:p="http://www.springframework.org/schema/p" 
  xmlns:context="http://www.springframework.org/schema/context" 
  xsi:schemaLocation=" 
    http://www.springframework.org/schema/beans  
    http://www.springframework.org/schema/beans/spring-beans-3.0.xsd  
    http://www.springframework.org/schema/context 
    http://www.springframework.org/schema/context/spring-context-3.0.xsd" 
> 
<bean 
  class="org.springframework.web.servlet.mvc.annotation.DefaultAnnotationHandlerMapping"/> 
<bean 
  class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter"/> 
 
<context:component-scan base-package="gae.application"/>

<bean 
  id="viewResolver" 
  class="org.springframework.web.servlet.view.ResourceBundleViewResolver"/>

</beans>

The Deployment Descriptor

web.xml

<?xml version="1.0" encoding="UTF-8"?> 
<web-app 
  id=2gae-application" 
  version="2.4" 
  xmlns="http://java.sun.com/xml/ns/j2ee" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee  
      http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"/> 
  <display-name>gae-application</display-name> 
  <servlet> 
    <display-name>Dispatcher Servlet</display-name> 
    <servlet-name>Dispatcher-Servlet</servlet-name> 
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> 
    <init-param>
      <param-name>contextConfigLocation</param-name>
      <param-value>/WEB-INF/WebCtx.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping
    <servlet-name>Dispatcher-Servlet</servlet-name>
    <url-pattern>/bo/*</url-pattern>
  </servlet-mapping>
  <welcome-file-list>
    <welcome-file>index.jsp</welcome-file>
  </welcome-file-list>
</web-app>

The GAE Deployment Descriptor

appengine-web.xml

<?xml version="1.0" encoding="UTF-8"?> 
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0"> 
<application>gae-application</application> 
<version>beta-1</version> 
<system-properties> 
<property 
  name="java.util.logging.config.file" 
  value="WEB-INF/classes/logging.properties"/>
</system-properties> 
</appengine-web-app>

The Maven Archetype

At the end of my experiment I pack my template application in a Maven archetype. I is enough to add the following pom.xml file to the project:

<?xml version="1.0" encoding="UTF-8"?>
<project
  xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
    http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>cloud.gae.application</groupId>
  <artifactId>gae-application</artifactId>
  <version>1.0</version>
  <packaging>war</packaging>
  ...
  <build>
    <finalName>${artifactId}</finalName>
    ...
  </build>
</project>

Now type mvn archetype:create-from-project to generate the archetype. Change the directory ( cd target/generated-sources/archetype ) and execute mvn install .
Using mvn archetype:generate now I may generate a GAE application ready to use. It does nothing but as starter kit for GAE applications is not bad.

URL Shortening Service

A URL shortening service is not essential for the most part of the applications in the WEB and may be dangerous because the surfer doesn't see what lies behind the short URL. But if I would like to publish something on a short messaging system like Twitter I need a very short URL.

The solution would be to register a very short domain like goo.gl and to run my own shortener, but for my application the additional effort is out of scope. So I need to connect to an existing URL shortening service.

The beginner has a first obstacle: where to find an inexpensive reliable URL shortening service. Using a search engine I did found a lot of service which are also free of charge, but no way to proof if the are also reliable.

Service Requirements

Since I want to generate the short URL into the background I need a service with API support. Additionally stating from the short URL I want to obtain the original (long) URL.

The candidates

• http://bit.ly
• http://budurl.com/
• http://cli.gs
• http://is.gd
• http://snipurl.com
• http://tinyurl.com
• http://zi.ma

I didn't want to check all the services, for instance four of them are enough.

bit.ly

The service is free of charge, but  requires a registration and checks each request using an API key.

• Homepage with a clear description of the service.
• Simple well documented API.
• Shorten and expand functions.
• JSON and XML formatted response.
  
• Simple but nice statistics summary.
• Blog

I like bit.ly and is currently my first choice.

is.gd

The service is free of charge and doesn't require a registration.

• Homepage with a clear description of the service.
• Very simple API with documentation.
• Shorten only function.
• Plain text (URL only) response.

This service is very spartan; I noticed that after two or three days the short URL was different. If a very short URL is required id.gd may be a good choice. 

snipurl.com

The service is free of charge, but requires a registration and checks each request using an API key.

• Homepage with a clear description of the service.
• Simple well documented API (Firefox doesn't display the page and with the MSIE there are some curious side effects, try with is-there-a-snipr-snipurl-api).
• Shorten and expand functions.
• XML and plain text (URL only) formatted response.
• Simple statistics but searchable statistics.
• Help

SnipURL exists since the year 2000, which is a sign of reliability. I miss the JSON formatted response, but the service is good.

tinyurl.com

The service is free of charge and doesn't require a registration.

• Homepage with a clear description of the service.
• Very simple API.
• Shorten only function.
• Plain text (URL only) response.

The service is very spartan, but the short URL is the same over the time.

Test the services

I did test this services for a while using a very simple HTTP client.

Client Requirements

The client have to be essential, but should be able to:

• retry the connection three times before to give up.
• follow the redirects (max three levels of indirection).
• support GET & POST.
• support the secure socket layer (SSL, nice to have).

Implementation

I use a very simple implementation done in Java and based on the HttpURLConnection, the following snippet shows the essential part of the code:

...
URL url = URL("http://myserver.com");
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod("GET");
conn.addRequestProperty("User-Agent","FastShortGood/1.0 Java/6");
...
// Extract the character set from the "Content-Type"
// header using a regular expression. If none available
// assume "UTF-8"
...
StringBuilder buf = new StringBuilder();
InputStream is = conn.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is, charset));
String line;
while (null != (line = br.readLine())) {
  buf.append(line).append("\n");
}
String body = buf.toString();

// Extract the short URL from response body using a
// regular expression.

For the complete implementation I need about 300 lines of code. The most of them handles exceptions and HTTP respose codes. The lightweight client works well with all of the shortener services.

Receiving Emails with the Google App Engine

Last year, in November, I did explore for the first time the Google App Engine for Java (GAE/J). One of the feature my application provides is to accept an email and translate it to an RSS 2.0 item.
The first version of the receiver was done using JavaMail. I did experiment some troubles with the "Quoted-Transfer-Encoding", sometimes the content was broken.
Using the current version of the GAE/J (1.3.0) the emails content received from Gmail or from Thunderbird is correct and the channel seems to be reliable.

Mime4J

An alternative for receiving email is the James Mime4J project.
The library doesn't have dependencies and works on the GAE/J. It is quite simple to use; instead of:

import java.io.InputStream;
import java.util.Properties;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
...
InputStream input = request.getInputStream(); // ServletRequest
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
MimeMessage message = new MimeMessage(session, input); // MessagingException
...
  

you have to use:

import java.io.InputStream;
import org.apache.james.mime4j.message.Message;
import org.apache.james.mime4j.message.MessageBuilder;
import org.apache.james.mime4j.parser.ContentHandler;
import org.apache.james.mime4j.parser.MimeStreamParser;
import org.apache.james.mime4j.storage.MemoryStorageProvider;
...
InputStream input = request.getInputStream(); // ServletRequest
Message message = new Message();
ContentHandler handler = new MessageBuilder(message, new MemoryStorageProvider());
MimeStreamParser parser = new MimeStreamParser();
parser.setContentHandler(handler);
parser.parse(input); // MimeException
...

The result seems to be better than the one which uses JavaMail. The mails received from the MS Outlook are readable. It remains a small problem with the encoding which I will investigate later this year.

Incoming Mail Syntax

The syntax of the incoming mail was changed:

description
line 1
line 2
...
line n
end

link
http://myserver/mylink
end

categories
category 1
category 2
...
category n
end

Each command block starts with a keyword and ends with end

Informations

The email address of the Mime4J service is mime4j@example-rest.appspotmail.com, the JavaMail based receiver still works with rss2@example-rest.appspotmail.com
The application URL is example-rest.appspot.com.