A Web Form Based PDFs Merger

Task: Provide a web-based application allows users to merger multiple files. Application is running in Grails framework.

Analysis: This problem can be split into three sub-tasks and be solved in three steps: upload multiple pdf files, retrieve submitted files and merger multiple pdfs.

Step1: Upload Multiple PDFs Files

Upload and process multiple files from a Web Form is not a trivial task because file input element allows uploading only one file at a time. Inspired by StickBlog's excellent post: Upload multiple files with a single file element. I decided to use Javascript instead of Applet to achieve this goal. I modified StickBlog's code to accommodate my needs. The user interface pdfmerger.gsp has the following element:

<g:form action="merge" method="post" enctype="multipart/form-data">
<input id='myfile' type='file' name='' onChange="addElement()"></input>
<input type="submit" value="Submit">

<br>Files list (Please note maximun number of uploaded files is 5):
<!-- This is where the output will appear -->
<div id="filesList"></div>
</g:form>

The event onChange of file input is captured. Each time a file is selected, the Javascript function addElement in script.js is invoked

var new_row = document.createElement('div' );

and a new <div> element is created and three elements: a text input box, a button and a file input box are appended to it. The text input box is just for display the file name. You can use other element for this purpose too:

var new_row_input =document.createElement( 'input' );
new_row_input.type = 'text';
new_row_input.name = "ins_" + (childs.length + 1)
new_row_input.value = element.value;

The button is used to delete a corresponding uploaded file if it is clicked on:

var new_row_button =document.createElement( 'input' );
new_row_button.type = 'button';
new_row_button.onclick = function (){
...
...
}

The file input box stored the uploaded files and will be submitted to server, and we like to make it invisible:

var new_row_file_input =document.createElement( 'input' );
new_row_file_input.setAttribute ('name','file_' + count);
new_row_file_input.setAttribute ('id','file_' + count);
new_row_file_input.value = element.value;
new_row_file_input.style.opacity = 0;

Finally, the newly created <div> element is appended to <div>with id"file_list " in pdfmerger.gsp:

new_row.appendChild(new_row_input);
new_row.appendChild( new_row_button );
new_row.appendChild(new_row_file_input);

target_list.appendChild (new_row);

The complete Javascript can be found here.

Step 2: Retrieve Submitted Files

Submitted files are retrieved and processed on server side in acontroller called PdfmergerController.java.In Grails, retrieving files is very easy by using build in Spring file system.We want to retrieve all submitted files and stored into an ArrayList for furtherprocessing.

for (i in 0.. max_num-1){
def file_name = "file_" + i;
def f = request.getFile(file_name);

if(f!=null && !(f.isEmpty())){
//println "file content type " + f.getContentType()
FileInputStream ins = f.getInputStream()
pdfs.add(ins);
}
}

Step 3: Merger Multiple PDFs

Once we have all submitted files in the list pdfs, we are ready to mergerthe PDFs. I just adopted the source code of method "concatPDFs" from Abhi'spost. It works so well with my system. The source code ofPdfmergerController.java can be found here.We added a beforeInterceptor to validatefiles content type before request is processed further.

Commentary

1. If server side processing is done ina J2ee environment, a third party library is needed since Java Servlet and Jspdo not have building mechanism to handle web form based file uploading. I hadApache Jakarta CommonsFileUpload package. It is an open source and can be downloaded from Apache Jakarta CommonsFileUpload project. Another package Apache Jakarta Commons IOproject is also needed internally by FileUpload package.

2. In step 3, we do file content typecheck on server side. However, it is also a good idea to validate content typeat client side when file is uploaded.

3. I just test the sample applicationwith IE and Firefox and no other else.

References

1. Uploadmultiple files with a single file element(StickBlog)
2. MergePDF files with iText (Abhi on Java)
3. Grails onlinetutorial
4. Tutorial: iText ByExample
5. ApacheJakarta Commons FileUpload project
6. Apache Jakarta Commons IOproject

Grails -- Using Embedded Derby Database

Apache Derby is a relational database implemented in Java. It is light-weighted and can be easily embed it in any Java-based solution. Here is a summary of using Derby Embedded JDBC driver within Grails framework. I used grails-0.3.1 for the time being.

Section 1: Install Software:

1.Install Apache Derby (v10.1.3.1)

Follow the tutorial from http://db.apache.org/derby/papers/DerbyTut/index.html to download and install software. (Note: ij tool is great to run SQL query from command line).

2.Install Grails (v 3.0.1)

The installation instruction can be found here. Set up environmental variables as described in the tutorial.

Section 2: Configure Grails for Using Embedded Derby Database

Inside $GRAILS_HOME, create sample application.

1. Create new Grails application by run command grails create-app, set up corresponding application name as myTest. Run command “grails create-domain-class”, set up domain name as “Book”.

2. Copy derby.jar and derbytools.jar from $DERBY_INSTALL to myTest/lib.

3. Configure data sources. In myTest/grails-app/conf, there are three data source files:

• DevelopmentDataSource.groovy
• ProductionDataSource.groovy
• TestDataSource.groovy

We need change the settings to let application talk to Derby instead of default Hypersonic database. A sample setting in ProductionDataSource.groovy would look like:

class ProductionDataSource {
boolean pooling = true
String dbCreate = "update" // one of 'create', 'create-drop','update'

##note: this setting point to a embedded db in /opt/db-derby-10.2.1.6-bin.
##The file could be anywhere that the application can reach
String url = "jdbc:derby:/opt/db-derby-10.2.1.6-bin/derbyDB"

String driverClassName = "org.apache.derby.jdbc.EmbeddedDriver”

String username = ""
String password = ""
}

4. We need another file in myTest/grails-app/hibernate/ called hibernate-dialects.properties which looks like:

DerbyDialect=Apache Derby

This file is required especially for Derby DB but not for Postgresql as I know.

Section 3: Run Application:

1. Then we are ready to run the application, execute the command:

grails run-app.

If everything goes right, we should be able to launch the application from http://localhost:8080/myTest.

2. To test the application by create some new records.

3. To verify whether the records are in Embedded Derby database, user Derby ij tool to run SQL query against our database prodDB. Please note, Derby won't allowed access from multiple applications for embedded model, so we have to shut down our application in order to run ij tool.

References

1. Apache Derby Tutorial (http://db.apache.org/derby/papers/DerbyTut/index.html)
2. Apache Derby Downloads (http://db.apache.org/derby/derby_downloads.html)
3. Grails Installation Instructions http://grails.codehaus.org/Installation

Grails -- Interceptors in Grails

Introduction

An Interceptor is one of the key components in Aspect-Oriented Programming (AOP). It is defined to intercept method invocations, constructor invocation and field access. In grails, there are only two types of interceptor: beforeInterceptor and afterInterceptor. They both are action interceptors used inside controllers.

Examples

befoerIntercetor

This simple interceptor intercepts an user's request and checks uploaded file content-type. If it is not a PDF file, then it redirects the user to a previous view and presents an error message and returns false to caller. The innovation of the method is terminated; If yes, it continues the method call and trigger an action. It is very easy to use, no any configuration is needed.

class myController {
def update = {/** do something here **/}
def save = {/** do something here **/}

//This line defines which actions is going to be intercepted.
def beforeInterceptor = [action:this.&validateFileType,only:['update', 'save']]

def validateFileType = {
def myFile= request.getFile("myFile") //get uploaded file
if (myFile!=null && !(myFile.isEmpty()))
{
def contentType = f.getContentType()
if (contentType!="application/pdf"){
flash.message = "File must be a valid PDF file"
redirect(action:create) //if fails, redirect to previous page.
return false //This will terminate method call
}
else {
...
...
return true
}
}
else {
...
...
return true
}
}
}

In the line

def beforeInterceptor = [action:this.&validateFileType,only:['update', 'save']]

"only" is called “condition”. The condition of an interceptor can be either "only" or "except". If the condition is not defined, then all actions in the controller will be intercepted.

afterInceptor

The second interceptor is afterInterceptor. Unlike beforeInterceptor, afterInterceptor gets invoked after an action has executed and before response gets back to client. We can process response object inside afterInterceptor before it gets back to clients, for example, writing logs, compressing files, sending emails, etc.

The usage of afterInterceptor is very similar to that of beforeInterceptor. In my previous posting Grail -- How to Send Out Email After Some Actions Have Been Finished, I have posted an afterInterceptor example. I would like to save some space here.

Commentary

In grails, it is quite easy to use action interceptors to intercepts process based on either request or response. And it is also quite flexible to define which actions would trigger interceptors. No any XML configuration needed, which is the promise of coding by convention.

However, there are still some limitations here. As I know for the time being, an interceptor can only intercepts actions from the same controller instead of multiple controllers. In other words, no interceptor chain is supported in Grails current releases. Despite of this, I still want to say: the interceptors are so cool!

References

Grails Tutorial#Controllers (http://grails.codehaus.org/Controllers)

Grails -- How to Add Binary PDFs Into a PdfDocument Using iText

We created an application using Grails framework. This application would allow registered users to submit application forms in PDF format. For some special reasons, we have to store all the PDFs into a relational database. Then we need to create a controller, such that the admin user is able to generate a PDF report which includes all the application forms that users had submitted.

The key issue for this task is to retrieve PDF files from the relational database and merge them into a single PDF file. Inspired by the post by Abni, we solved the problem by using iText in the following steps:

Step 1: Create a PdfDocument and PdfWriter:

Document document = new Document(PageSize.A4, 50, 50, 50, 50);
PdfWriter writer = PdfWriter.getInstance(document, response.getOutputStream())

Step 2: Retrieve a binary pdf file from database and write into a byte array:

Suppose I have a domain class like this:

class Application {
Long id
ong version

String username //Primary key
byte[] application
...
...
}

In grails, it is easy to retrieve a pdf file out of database and write into a byte array , eg:

def app = Application.findByUsername(userName) //here userName is the primary key
if (app != null && app.application.size() > 0){

def byte[] app_letter = app.application
...
...
}

Step 3: Create a PdfReader from a byte array which represents a PDF file:

PdfReader pdfReader= new PdfReader(app_letter)

Please note that the byte array must represent a valid PDF file, otherwise, an exception is thrown.

Step 4: Create a page from PdfReader and add the page to PdfDocument:

PdfImportedPage applicationPage
PdfContentByte

//the pdf file may contain more than one pages
for (pageNumber in 0..pdfReader.getNumberOfPages()-1){
pageNumber++

applicationPage = writer.getImportedPage(pdfReader, pageNumber) //Create page from PdfReader

cb.addTemplate(applicationPage, 0, 0) // add page to PdfDocument
....
....
}

The snippets above just show how to retrieve one PDF file from database and add it into a PdfDocument. For multiple PDF files, we can do the same thing for each of them and update page number correspondingly.

Discussion:

1. In step 3, the byte array used to create PdfReader must represent a valid PDF file, otherwise, an exception throws. The program should handle the exception.

2. PDF files are all hold in memory. If file number is huge, then we need to consider memory issues.

Reference:

1. Merge PDF files with iText (Abhi on Java)
2. Grails online tutorial
3. Tutorial: iText By Example

Grails -- How to send out an email after some actions have been done?

In my work, I need to send out a notice to the administrators each time a user submit an application. There are several ways to do this. The example here just demonstrates how easy it can be done by using "afterInterceptor" just in couple of steps:

Step 1: Follow the tutorial posted here to configure a couple of beans which will be used for sending email.

In spring/resources.xml, add the following configurations:

<beanid="mailSender"class="org.springframework.mail.javamail.JavaMailSenderImpl" >
<property name="host">
<value>mailServer.abc.com</value>
</property>
</bean>

Step 2: Copy three jar files into the application' local lib: activations.jar, mail.jar and javaee.jar. These jar files are needed by org.springframework.mail.javamail.JavaMailSenderImpl. which is registered in the above spring/resources.xml file.

Step 3: Add the following code into the controller that I want to intercept. Because I want to attach a PDF file, so I used MimeMessageHelper .

class ApplictionController {

/** the mailSender properties is defined in the spring/resources.xml and are automatically injected by Grails**/
MailSender mailSender

def afterInterceptor = [action:this.&sendMail,only:['save']]

def sendMail= {
MimeMessage message = mailSender.createMimeMessage();
MimeMessageHelper helper = new MimeMessageHelper(message, true, "UTF-8");

//recipient of email could be a list of email addresses
String [] to = new String[2]
to[0] = "address1@gmail.com"
to[1]= "address2@gmail.com"

helper.setTo(to)
helper.setSubject("It is a test")
helper.setText("check out this PDF file")

//attach a file
FileSystemResource file = new FileSystemResource(new File("/path/to/myApplication.pdf"));
helper.addAttachment("myApplication.pdf", file);

//send mail
mailSender.send(message)
}
...
...
...
}

Commentary:
1. One interceptor can be used to intercept more than one actions, for example:

def afterInterceptor = [action:this.&sendMail,only:['save', 'update']]

The Interceptor will execute each time after "save" or "update" are finished

2. The attachment could be binary data from the database. In this case, read data into a byte array and then create a ByteArrayResource from the byte array. The ByteArrayResource can be attached too.

3. We can intercept before some actions get executed basically the same way but using "beforeInterceptor" instead.

Issues: It turns out a very simple solution once I figured out how to do it. However, the solution above just intercepts actions based on a instance of one controller. What if I want to intercept multiple controller using only one interceptor? Can we do this in Grails?