Data Processing

Data processing is the conversion of data into usable and desired form. This conversion or
“processing” is carried out using a predefined sequence of operations either manually or automatically.
Most of the data processing is done by using computers and thus done automatically. The output or
“processed” data can be obtained in different forms like image, graph, table, vector file, audio, charts
or any other desired format depending on the software or method of data processing used.

Fundamentals of data processing & how data is processed

Data processing is undertaken by any activity which require collection of data. This data collected
needs to be stored, sorted, processed and analyzed. This complete process can be divided into 6 basic
simple stages which are:

1. Data collection
2. Storage of data
3. Sorting of data
4. processing of data
5. Data analysis
6. Data presentation and conclusions

Once the data is collected the need for data entry emerges for storage of data. Storage can be done in
physical form by use of papers, in notebooks or in any other physical form. But now since the data
collection is large and number of operations needs to be performed for meaningful analysis, the data is
stored in digital form. Having the data into digital form enables the user to perform large number of
operations in small time and allows conversion into different forms. User can thus select the output
which best suits the requirement.

This continuous use and processing of data follows a cycle called as data processing cycle which
might provide instantaneous results or take time depending upon the need of processing data. The
complexity in the field of data processing is increasing which is creating a need for advanced
techniques.

Storage of data is followed by sorting and filtering. This stage is highly effected by the format in
which data is stored and further depends on the software used. General daily day and non complex data
can be stored as text files, tables or a combination of both in Microsoft Excel or similar software. As
the task becomes complex which requires performing specific and specialized operations they require
different softwares which are meant to cater the peculiar needs.

Storing, sorting, filtering and processing of data can be done by single software or a combination of
software whichever feasible and required. Data processing thus carried out by software is done as per
the predefined set of operations. Most of the modern-day softwares allows users to perform different
operations based on the analysis or study to be carried out. Data processing provides the output file in
various formats.

Different types of output files obtained as “processed” data

Plain text file – These constitute the simplest form or processed data. Most of these files are user
readable and easy to comprehend. Very negligible or no further processing is these type of files.

Table/ spreadsheet – This file format is most suitable for numeric data. Having digits in rows and
columns allows user to perform various operations like filtering & sorting in ascending/descending
order to make it easy to understand and use. Various mathematical operations can be applied when
using this file output.
Charts & Graphs – Option to get the output in form of charts and graphs is very useful and now
forms a standard features in most of the softwares. This option is very useful when dealing with
numerical values reflecting trends and growth/decline. Though there are ample of charts and graphs are
available to match diverse requirements there exists situation when there is need to have used defined
option. In case no inbuilt chart or graph is available then the option to create own charts i.e. custom
charts/graphs comes handy.

Maps/Vector or image file – When dealing with spatial data the option to export the processed data
into maps, vector and image files is of great use. Having the information on maps is of particular use
for urban planners who work on different types of maps. Image files are obtained when dealing with
graphics and does not constitute any human readable input.

Other formats/ raw files – These are the software specific file formats which can be used and
processed by specialized softwares. These output files may not be a complete product and require
further processing. Thus there will be need to perform multiple data processing.

Types of data processing

Manual data processing: In this method data is processed manually without use of machine or
electronic device. This methods might be accompanied with automatic method for completion of the
data processing.

Mechanical data processing – Data processing is done by use of mechanical device or very simple
electronic devices like calculator and type writers. When the need for processing is simple this method
can be adopted.

Electronic data processing – This is the fastest and best available method with highest reliability and
accuracy. Technology used is latest as this method uses computers and employed in most of the
agencies. The use of softwares forms the part of this type of data processing.

Types of data processing on basis of process/steps performed

1. Batch Processing
2. Real time processing
3. Online Processing
4. Multiprocessing
5. Time sharing

Data processing methods

Manual Data Processing – data is processed manually without using any machine or tool to get
required results. In manual data processing, all the calculations and logical operations are performed
manually on the data.

Mechanical Data Processing – data is processed by using different devices like typewriters,
mechanical printers or other mechanical devices.

Electronic Data Processing is the modern technique to process data. The data is processed through
computer; Data and set of instructions are given to the computer as input and the computer
automatically processes the data according to the given set of instructions. The computer is also known
as electronic data processing machine.
What makes processing of data important

Nowadays more and more data is collected for academic, scientific research, private & personal use,
institutional use, commercial use. This collected data needs to be stored, sorted, filtered, analysed and
presented for it to be of any use. This process can be simple or complex depending on the scale at
which data collection is done and the complexity of the results which are required to be obtained. The
time consumed in obtaining the desired result depends on the operations which needs to be performed
on the collected data and on the nature of output file required to be obtained. This problem becomes
more stark when dealing with very large volume of data such as those collected by multinational
companies about their users, sales, manufacturing etc.

The need of data processing becomes more and more important in such cases. In such cases data
mining and data management comes into play without which optimal results cannot be obtained. Each
stage starting from data collection to presentation has direct effect on the output and usefulness of the
processed data.

Utility software

Utility software is system software designed to help analyze, configure, optimize or maintain a
computer.[1] It is used to support the computer infrastructure in contrast to application software, which
is aimed at directly performing tasks that benefit ordinary users.

Although a basic set of utility programs is usually distributed with an operating system (OS), utility
software is not considered part of the operating system, and users often install replacements or
additional utilities. It provides additional facilities to carry out tasks which are beyond the capabilities
of the operating system.

Many utilities which might affect the entire computer system require the user to have
elevated privileges, while others, which operate only on the user's data, do not.

Types of utilities

System utilities

 Anti-virus utilities scan for computer viruses and remove them.

 Clipboard managers expand the clipboard functionality of an operating system .
 Memory testers check for memory failures.
 Package managers are used to configure, install or keep up to date other software on a computer.
 Registry cleaners clean and optimize the Windows Registry by removing old registry keys that
are no longer in use.
 Screensavers were desired to prevent phosphor burn-in on CRT and plasma computer monitors by
blanking the screen or filling it with moving images or patterns when the computer is not in use.
Contemporary screensavers are used primarily for entertainment or security.
 Network utilities analyze the computer's network connectivity, configure network settings, check
data transfer or log events.
 System monitors monitor resources and performance in a computer system.
 System profilers provide detailed information about the software installed and hardware attached
to the computer.

Storage device management utilities

 Disk checkers scan an operating hard drive and check for logical (filesystem) or physical errors.
 Disk compression utilities transparently compress/uncompress the contents of a disk, increasing
the capacity of the disk.
 Disk defragmenters detect computer files whose contents are scattered across several locations on
the hard disk, and move the fragments to one location to increase efficiency.
 Disk formatters prepares a data storage device such as a hard disk drive, solid-state drive, floppy
disk or USB flash drive for initial use.
 Disk partition editors divide an individual drive into multiple logical drives, each with its own
file system which can be mounted by the operating system and treated as an individual drive.
 Disk space analyzers provide a visualization of disk space usage by getting the size for each
folder (including sub folders) & files in folder or drive. showing the distribution of the used space.
 Tape initializers write a label to a magnetic tape or other magnetic medium. Initializers
for DECtape formatted the tape into blocks.

File management utilities

 Archivers output a stream or a single file when provided with a directory or a set of files.
Archive suites may include compression and encryption capabilities. Some archive utilities have a
separate un-archive utility for the reverse operation.
 Backup software makes copies of all information stored on a disk and restores either the entire
disk (e.g. in an event of disk failure) or selected files (e.g. in an event of accidental deletion).
 Cryptographic utilities encrypt and decrypt streams and files.
 Data compression utilities output a shorter stream or a smaller file when provided with a stream
or file.
 Data synchronization utilities establish consistency among data from a source to a target data
storage and vice versa. There are several branches of this type of utility:
 File synchronization utilities maintain consistency between two sources. They may be used to
create redundancy or backup copies but are also used to help users carry their digital music,
photos and video in their mobile devices.
 Revision control utilities can recreate a coherent structure where multiple users
simultaneously modify the same file.
 Disk cleaners find files that are unnecessary to computer operation, or take up considerable
amounts of space. Disk cleaner helps the user to decide what to delete when their hard disk is full.
 File managers provide a convenient method of performing routine data management, email
recovery and management tasks, such as deleting, renaming, cataloging, uncataloging, moving,
copying, merging, generating and modifying folders and data sets.

Miscellaneous utilities

 Data generators create a file of test data according to specified patterns.

 Hex editors directly modify the text or data of a file. These files can be data or an actual program.
 HTML checkers validate HTML code and check links.
 Sort/Merge programs arrange records (lines) in a file into a specified sequence.