Crucis Consulting Limited

Data capture

Daishik
14 Jul 2021
0

One of the most common requirements of a software application is to get users to enter data.

This could be a survey, an online form for a loan application, the need to complete a profile or purchase, or simply to login or verify identity. Or obtain consent to capture and hold even more data!

In the main, for business applications the online form is the mainstay of data capture.

So it's just a question of providing as many textboxes as are needed and getting users to fill them in, right?

Firstly there are a number of elements that are used in online forms and their appropriate use requires some thought. The most basic - the ones that have been in use since the early days of the web - are:

Textbox - for single line of text e.g. Your favourite planet. A special type of textbox is used for passwords or secrets.

Textarea - for multiline text e.g. Do you want to add any general comments?

Checkbox - for yes/no type answers e.g. Do you want to gift-aid your donation? Do you want to save your credentials? Checkboxes are also useful to select multiple choices from a small list of options, such as a survey: select all the services you have used in the past 3 months.

Radio-button list - single choice from a selection of options e.g. Which of our colours is your favourite?

Droplist - single choice from a longer selection of options. Only one option is visible at a time.

Select list - single or multiple selection from a selection. Multiple options can be visible at a time, and users can select multiple options as well.

Buttons - process an action on the form e.g. submit, cancel or search

Depending upon the type of data you are capturing, some elements may be more appropriate than others and the choice is not always obvious.

Advanced controls

There are various tools and techniques that make data capture better than simply using the basic components above. Browsers have over the years also come a long way in improving the data capture, as has the underlying HTML specification. The need to efficiently enter data from browsers on small touch devices such as smartphones has also driven these enhancements.

Modern web technologies go a long way to augment the original data capture elements, with better usability and opportunities for creativity. It is common now to see sliders (for numbers), auto-search (think postcode address search) and auto-complete list items. There are also far-improved data validation techniques to improve the quality of the data being captured.

For example, when was the last time you actually typed a date into a field? Most of the time there is a calendar control of some type which allows you to quickly find the date and the data gets entered automatically. This avoids issues with users entering invalid dates or getting their date formats incorrect on international sites.

Other kinds of control are also used to improve specific data capture scenarios. If the choice is over a particularly long list, designers may categorise the list and/or use checkboxes as a means to capture multiple selections, or to integrate it with an automatic search and filter, so the list adapts to the characters the user types.

For particularly large capture scenarios, the form may be sub-divided over multiple screens with a partial save mechanism between screens so users can leave the process and continue later.

Automated capture

We all know about the way cookies and link tracking is used to monitor our use of certain websites and even our general activity online. This type of data capture happens automatically and users are often oblivious to it.

Users of smartphones are also aware that their actual physical movement is also being captures in addition to their use use of websites and apps.

The use of automated data capture is controversial, but there are some instances where there can be good reasons to do so if it helps user experience or improves data accuracy. The important issue is it needs to be clear and consensual, and driven by a clear purpose.

What if you needed to capture some information about the user or their device that the user themselves might not know or find difficult to enter accurately themselves? What if there may be an incentive for the user to provide inaccurate data? For example, how many users know their own IP address or the precise version of their browser or operating system, or other technical details of their smartphone? This kind of information may be helpful from a customer support perspective, and are best captured - with the user's consent - automatically.

You can also capture some data automatically simply for convenience: We can capture the current date and time and perhaps user location automatically because it's quicker than asking the user to enter them manually.

Validation

It's one thing to get users to enter the data you wish to capture. Then there is the question of validation and pre-processing.

Validation is about ensuring that the data being entered satisfies your expectation of what that data should look like. For example, should a field be required/mandatory or optional?

Does it need to satisfy some pattern? When entering an email address or phone number, the underlying element would be a textbox. However, does what the user enters look like a legitimate email address or phone number? Does it have the correct pattern and number of characters - not too many or too few? Does it contain any illegal characters? In an email for example, you would expect an '@' character. For a phone number you would not.

In most cases validation happens in the browser as soon as the user enters the data. This is good because it tells users immediately and catches out inadvertent errors.

The next stage is to submit the data to the web server where the data will undergo further validation and checking before it gets saved. This is sometimes called pre-processing, business validation or business rule execution.

Whilst validation tends to be standard across applications, pre-processing can be as simple or complex or bespoke as you need it to be.

For example you may have a bespoke rule that requires every email to be unique on your system. So you want to check that an email address (which has been successfully validated) is not already in use by another user on your system. Your pre-processing rule would check your database to verify this.

Or perhaps you wish to check that the user has paid their membership fee for the current year before they can book a place on a workshop. Again, you can verify this before saving the booking.

If pre-processing fails, the user can be notified of the reasons and appropriate actions can be made available.

Summary

Capturing data effectively can involve much more than simply providing a form for users to fill in.

In businesses, users may be constantly interacting with the capture process and then reading back data throughout the day and the process needs to be as efficient and user-friendly as possible.

If you have a data capture or data management need, why not give us a call to discuss?