Batch Processing

Batch processing is the processing of transactions in a group or batch. No user interaction is required once batch processing is underway. This differentiates batch processing from transaction processing, which involves processing transactions one at a time and requires user interaction. While batch processing can be carried out at any time, it is particularly suited to end-of-cycle processing, such as for processing a bank's reports at the end of a day, or generating monthly or bi-weekly payrolls.

For large enterprises, batch processing became a normal way of data compilation, organization, and report generation around the middle of the 20th century with the introduction of the mainframe computer. The early mechanics of processing a batch involved feeding a computer a stack of punched cards that held commands, or directions, for the computer to follow. Herman Hollerith (1860-1929) is credited with developing the punch card around 1890 when he was employed as a statistician for the U.S. Census Bureau. It was this punch card that became the seed for widespread batch processing around 50 years later.

Batch processing jobs are run on regularly scheduled times (e.g., overnight) or on an as-needed basis. As an example, bills for utilities and other services received by consumers are typically generated by batch processing each month. Batch processing is beneficial because it is a cost-effective means of handling large amounts of data at once. One caveat: The inputs for the processing must be correct or else the results of the whole batch will be faulty, which would cost time and money.

A defining characteristic batch processing is a lack of human intervention, with few, if any, manual processes to kick it off. This is part of what makes it so efficient in modern times, but it wasn't always that way.

Batch processing started with punch cards, which were tabulated into instructions for computers. Entire decks, or batches, of cards would be processed at one time.

This practice dates back to 1890 when Herman Hollerith first created punch cards to process U.S. Census data. In his system, a card that he punched manually was read by an electromechanical device. Hollerith later formed a company that would become IBM.

Today, most batch processing functions are enabled without human interaction, and they are completed to meet specified timing needs. Some jobs are completed in real time with daily monitoring and reporting features; others are done immediately. Modern batch processing uses exception-based management alerts to notify the right people if there are issues. This allows managers the freedom to work without regularly checking in on batches, unless they receive an alert about a critical exception.

Exceptions are determined by a system of dependencies and monitors that are essential to the software. Dependencies are the event that triggers the batch processing to begin. This could be that a customer places an online order, or a system generated request for new supplies comes in.

Faster and Lower Cost

Without the need for data clerks to support its functioning, batch processing helps to reduce the operational cost that businesses spend on labor. It also doesn’t require any additional hardware beyond a computer. In fact, Batch Processing can reduce a company’s reliance on other expensive pieces of hardware. Without the possibility of user error, the results of batch processing are available fast and provide managers more time to spend on day-to-day operations.

Offline Features

Batch Processing systems can work offline, in the background, outside of regular business hours. This provides a convenient solution for businesses that don’t want a job like automatic downloads to disrupt daily activities.

Hands-Off Management of Massive Tasks

The exception-based notification system of modern batch processing makes it easier for managers to do their job without worrying if batches are being completed. If there’s an issue, notifications are sent to the right people to solve it. Managers can take a hands-off approach and trust their batch processing software to do its job.

There are a few things that business owners should consider before implementing batch processing systems.

Deployment and Training

Like many technologies, a degree of training is required to manage batch processing systems. Managers will need to learn what triggers a batch, how to schedule them, and what exception notifications mean, among other things.

Debugging Can Be Tricky

Debugging batch processing systems can be complex. If there isn’t someone within your organization that has a deep understanding of these systems, there may be a need for an outside consultant to assist.

Cost

Batch processing infrastructure can be an expensive capital outlay. For some businesses, the costs may seem unfeasible.