Acrobat User Community

Form submit /eMail demystified

By Thom ParkerNovember 4, 2013

Tip – Submitting Forms, How and Why Form Submit/Email Demystified by Thom Parker, Software Developer/Adventurer, WindJack Solutions, Inc.

Scope: Acrobat XI
Skill Level: Beginner
Prerequisites: Acrobat Forms

The form technology in Acrobat has built-in features for common form operations such as resetting, printing and submitting forms. Resetting and printing are fairly straightforward and intuitive actions. But form submission is a different kind of animal. It is among the most convoluted and frustrating of all form features. In this article, I will attempt to demystify the form submission process and make this vital process accessible to even the most non-technical of users. We'll cover this process as it relates to PDF forms displayed in Acrobat or Reader, on regular old computer systems (PC or Mac). The techniques and technologies discussed here are not yet available on mobile devices.

Why Submit is so different from other form actions

Most form operations happen entirely inside the form environment. For example, a form data reset action only touches the fields on the form. There are no security issues or implementation difficulties here. It's a purely safe and self-contained operation. On the other hand, operations like printing a form require going outside of the form environment-- in this case, to access print drivers on the user's computer. Printing is not a very risky operation, but nonetheless it's still touching something outside of the form, so Acrobat pops up a warning message letting the user decide whether or not to allow the print. After all, you wouldn't want random PDF files printing stuff without your knowledge or control. There are also potential implementation issues here. For example, there has to be an actual printer hooked up to the system and it has to be turned on. These are things Acrobat has no knowledge or control over, so you can see that even with something as simple as printing, there are potential problems when going outside of the form environment.

Form submission is an entirely different kind of animal from these other operations. It not only has to go outside the form environment, it has to go completely outside the user's computer. There are different ways to perform a submit operation, several elements that must be in place to make it work, and many different ways it can go wrong. In addition, nearly all of these things are totally outside of Acrobat's control.

How form submission works

Data submission is all about getting the form data from the person who fills out the form to the person who manages the form. In the old days, you would use a pen or pencil to fill out an actual paper form and then snail-mail or even physically hand the form over to the person who manages the form. Today, of course, we have many other ways to move form data from place to place.

To submit PDF form data electronically, the form designer adds a submit button to the PDF. Acrobat has both built-in button functionality for performing data submission and a JavaScript function which provides many more options and greater flexibility. The built-in functionality is also in the official PDF ISO specification. So, ideally it should be implemented in every PDF viewer. That is not quite the case yet, partly because there is no standardization on exactly how the submit operation is implemented. But to continue with Acrobat, in order to perform the submit operation Acrobat needs two pieces of data, a destination and a format in which to transmit the data.

Traditionally (in Acrobat/Reader), a PDF form submits data through one of two data-transfer mechanisms, email or HTTP (the standard web page transfer protocol). As such, the submit destination can be either an email address or an URL to a server script. Both of these submission types have been around for a very long time (since Acrobat 3). Of course, form submission and data handling have never been truly intuitive or easy. In an attempt to make this task easier for ordinary users, Adobe introduced a Forms Distribution Wizard in Acrobat 8; and in Acrobat/Reader 9, they introduced a third data transfer mechanism- submission to the Acrobat.com server. All these mechanisms attempt to hide the messy technical details from the user. I'll discuss each one in turn.

Email submission

Email seems to be by far the most popular way to send form data. It has certainly been written about more than submitting by other means. Here are some links to articles about, or that cover some aspect of email form submission:

I'm sure that email is popular because most people are familiar and comfortable with sending and receiving emails. Anybody who has a computer and an Internet connection already has an email account. This method doesn't require any special programming or servers. Ideally, after filling the form, the user simply presses the submit button and the form data is magically attached to an email and delivered back to the form manager, simple and sweet. Unfortunately, it doesn't always work out so nicely. Email submission is the most problematic solution for both ends of the transfer, sending the data on the user side and collecting the data on the receiving end. And here's the reason why.

Acrobat doesn't actually send the email. It hands the email to the operating system, and the system sends the email through the default email client. The default email client is the program you use to send and receive emails. It's the one that pops up automatically whenever something happens on your system that has to do with email. Both the Windows and Macintosh operating systems have the ability to register an email program as an operating system service. Once the email client is registered, any program can use it, and this is exactly what Acrobat is doing.

The problem with this approach is that many people do not have the local email client set up, or do not have it set up properly. In many cases, this is because they use a Web- based email service such as Gmail, Hotmail, and Yahoo mail. Email may seem simple to the user, but it is an awkward topic for a desktop program like Acrobat. So, starting with version 9, Acrobat tried to make email more reliable by giving the user options for saving the form and then manually sending it through the user's preferred method (Figure 1). Not an ideal solution.

Email Options

Figure 1 — Acrobat 9 Email options shown to the user, for forms enabled by the Distribution Wizard

Now, Acrobat/Reader XI has a much better solution. It provides many more options for allowing the user to connect to different types of email services. Figure 2 shows the dialog that Reader XI displays when the end user clicks on a form submit button.

Email Help

Figure 2 — Reader XI provides several email submit choices

Data collection is the other side of data submission. Once you get the filled forms back in your email inbox, you have to do something with them. Since PDF forms were introduced, Acrobat has always provided some data handling options. But in Acrobat 8, Adobe started adding much more sophisticated options, such as merging form data into a spreadsheet. Figure 3 shows where these options are accessed in Acrobat XI.

Data Options

Figure 3 — Generic options in Acrobat XI for compiling and handling form data

Acrobat 8 was also the first version to offer form distribution and a data collection file. This particular technique has become much more sophisticated in Acrobat XI. The Distribution Wizard can make form data much easier to deal with, at least for simple forms. Here are some articles that discuss dealing with the data on the receiving end of the forms data workflow (Unfortunately there are not many new articles on data collection for Acrobat XI, but not to worry, it's pretty much the same as it was in Acrobat 8).

When it works, email submission is a very easy and intuitive way to handle form data. However, besides the issue of dealing with users that do not have a proper email setup, it's important to keep in mind that handling the return data by email is usually a manual process. This means that email submission is an inherently low-volume methodology. It's actually quite perfect for smaller groups of known users, but for a large number of random users it may be unwieldy, inefficient, and time consuming.

HTTP submission

HTTP submission is very similar to how web pages are requested by a web browser. Acrobat packs up the form data, or the entire form, into what's called an HTTP Request and then sends it out through the user's internet connection. As long as the user has an internet connection, no additional software or setup is needed on the system. This kind of submission is much more reliable and transparent than email.

The HTTP Request is sent to a web server. The server destination that receives the data has to be a program that can extract the form data from the HTTP Request and do something useful with it. Usually, this program is a server script written in one of the common server scripting languages such as Perl, PHP, .Net, ColdFusion, Java, etc., but it could also be a custom server designed specifically for handling PDF forms.

Because data is handled automatically by a server, this submission method can easily deal with large quantities of data and is also very flexible. Sever scripts can be written to intelligently handle the data by validating it, automatically transferring it into a database, sending a response back to the user, sending alerts to the forms manager, or any other action necessary for a particular data workflow.

The obvious difficulty with HTTP submission is the server-side handling. Anyone can create a form with email submission for no extra cost. And I mean literally anyone. No technical knowledge necessary. The email infrastructure is well known and already exists for the majority of computer users. HTTP submission is a completely different animal, someone has to write and install the server-side program. If you want custom functionality, you'll need to do it yourself, and this is exactly the path taken by many larger organizations. To make this task a little easier, ColdFusion (a server programming language distributed by Adobe) has built-in features specifically for handling PDF forms. Adobe also sells a very sophisticated forms server. And for smaller organizations, there are third-party companies that provide data-handling services. Whatever route you go, this methodology requires extra effort and/or cost.

The Form Distribution Wizard

From the discussion so far, you can see that setting up a form-data-handling process is not necessarily an easy thing to do, especially if you want to make things easy for the end user. It's not sufficient to simply put a submit button on the PDF. The returned form data also has to be managed. And if you have more than one form, the process becomes increasingly difficult to manage. To make the whole process a bit more automated and accessible to users, Adobe added a set of form data handling features to Acrobat 8, and continues to improve them with each new version. The entry point into using these features is the Form Distribution Wizard (shown in Figure 3).

There are three parts to this form-handling process:

  1. Preparation for distribution to users.
  2. Collection and organization of returned data.
  3. Tracking and managing the form data process.

When the Distribution Wizard is activated, it makes a copy of the original form and then prepares this copy for distribution by adding a submit button (if necessary), setting up the submit destination, and enabling the form for Save Rights in Reader (no longer necessary if the form is used in Adobe Reader XI). The Distribution Wizard always sets up the form to submit the entire PDF by email. This methodology is a bit narrow and inflexible, but it is consistent and allows for the widest possible range of form variations. For example, it handles digitally signed forms.

The wizard also creates a collection file. This is a variation of the original form that's set up to literally collect the returned forms. Special information is added to both this file and the distributed form so that Acrobat automatically knows to associate the two. Ideally, when form managers receive an email with a filled out form, they simply click on the email file attachment and the returned form is added to the collection file.

The Distribution Wizard also registers the form to the Forms Tracker. This is a special piece of software that ships with Acrobat. When you install Acrobat, the Forms Tracker is installed and set up to run in the background. It's a control panel for managing distributed forms. The tracker in Acrobat 8 is a bit primitive. All it does is list and provide access to the registered forms. The tracker in Acrobat XI is much more sophisticated, provides information at a finer granularity, and provides access to more parts of the workflow process.

Finally, after all of this setup, the Distribution Wizard emails the form to a set of users. It's not necessary for the Distribution Wizard to email out the form, this is just a convenience. The form can be posted on the web or emailed out at a later time.

The Distribution Wizard is supposed to make form-data handling easy and it does accomplish this to a large extent. It sets up the form for every step of the data-handling process, from distributing the form to collecting returned data. However, in Acrobat 8 it is purely an email-based process, and it has exactly the same issues discussed in the email submit section above. It is a low-volume methodology, and can be problematic if the user does not have a proper email setup. On top of this, the collection end does not always work smoothly. It's possible to miss data or to double-enter data into the collection file.

Summary

Data submission is only part of the process of handling form data. The data also has to be collected and compiled into some useful form. In the simplest case, the whole form can be submitted by email back to the form manager who prints it out and puts it in a file cabinet, just like the old paper process. In the most sophisticated scenario, data is submitted to a server that handles an advanced data-flow process. To help with this process, Adobe added a set of form data tools to Acrobat 8 in the form of the Form Distribution Wizard. Acrobat XI also made major improvements in all areas of data handling, and Reader XI added major improvements to the way email submission is handled for all forms. All of these variations have limitations, potential problems, and costs. It's up to the form designer to understand these variations and choose the one that best fits their needs.