Analysis of Penetration Testing and Vuln
Analysis of Penetration Testing and Vuln
Keywords 1. Automatic
Penetration Testing, Pen Tester, Cyber Security, Vulnerability The automatic penetration is often chosen when cost is a key
Assessments, Risks, Attacks. factor. Due to the free software availability of many penetration
tools, a company could choose to have the penetration performed
I. Introduction by this method. Also, commercial tools that could be used have
There are two types of penetration: a cost associated with them; however, this tool cost could be
spread out and would still be a less costly solution than manual
1. Internal penetration.
This testing is often performed from different network access However, the learning curve for each penetration tool is usually
points that include both the physical and logical segments; this much higher, and the knowledge required and experience in doing
provides a more detailed view of the security. such work demands the skills of an expert.
2. External 2. Manual
This testing has its focus on the infrastructure components, servers, Manual penetration is usually chosen to give an independent
and the related software of the target. It also provides a detailed assessment of the penetration. Normally an external company
analysis of the information that is available from public sources, that is experienced in the field and does it on a regular basis, with
such as the Internet. Enumeration of the network is also performed a good track record, is chosen. Regulation requirements could
and analyzed. The filtering devices, such as firewalls and routers, make this the only alternative a company has.
are also scrutinized for their vulnerabilities. Finally, the impact
C. Vulnerability Assessment work as normal during and after a test; ensuring high availability
A vulnerability assessment is the process of identifying, and minimizing the disruption to business processes. This means
quantifying, and prioritizing (or ranking) the vulnerabilities that the systems may not have been fully penetrated in order
in a system. Examples of systems for which vulnerability to determine the degree of risk these vulnerabilities may pose.
assessments are performed include, but are not limited to Hackers on the contrary, do not care if availability of a system
information technology systems, energy supply systems, water goes down and will attack it to achieve their set goals by any
supply systems, transportation systems, and communication available means.
systems. Such assessments may be conducted on behalf of a Usually, large corporations look at hiring a penetration tester
range of different organizations, from small businesses up to large to minimize any future damage or information leakage from a
regional infrastructures. IT infrastructures are assessing, on an potential hacking incident. There is also increasing pressure for
ongoing basis, the real risks presented by system vulnerabilities. corporate organizations to comply to external standards (e.g.,
The task of correctly assessing the real security risks associated Sarbanes-Oxley, HIPAA, PCI DSS, ISO 27001) which usually
with a seemingly endless stream of vulnerability and patching require or recommend some form of security review (Bentley, L.,
reports is a critical and time-consuming activity for IT staffs. 2006). This does mean that these can occasionally lead
However IT professionals understand that despite their best to a simple security exercise with a ‘tick in the box’ approach and
efforts, vulnerabilities may still present significant security risks therefore limiting the penetration tester to conducting a simple
for their companies. During research and in-depth interviews, IDC vulnerability assessment.
found compelling reasons why IT executives and team members
must adopt penetration testing as an integral part of their security I. Penetration Testing vs. Vulnerability Assessment
and vulnerability management (SVM) processes and programs. The main focus of this paper is penetration testing but there is often
Penetration testing enables users to: some confusion between penetration testing and vulnerability
assessment. The two terms are related but penetration testing has
D. Intelligently Manage Vulnerabilities more of an emphasis on gaining as much access as possible while
Penetration testing provides detailed information on actual, vulnerability testing places the emphasis on identifying areas that
exploitable security threats. By performing a penetration test, an are vulnerable to a computer attack. An automated vulnerability
organization can identify which vulnerabilities are critical, which scanner will often identify possible vulnerabilities based on service
are insignificant, and which are false positives. banners or other network responses that are not in fact what they
seem. A vulnerability assessor will stop just before compromising
E. Avoid the cost of Network Downtime a system, whereas a penetration tester will go as far as they can
Recovering from a security breach can cost millions due to IT within the scope of the contract. It is important to keep in mind
remediation efforts, lost employee productivity, and lost revenue. that you are dealing with a ‘Test.’ A penetration test is like any
Penetration testing allows an organization to prevent this financial other test in the sense that it is a sampling of all possible systems
drain by identifying and addressing risks before security breaches and configurations. Unless the contractor is hired to test only a
occur. single system, they will be unable to identify and penetrate all
possible systems using all possible vulnerabilities. As such, any
F. Preserve Corporate Image and Customer Loyalty Penetration Test is a sampling of the environment. Furthermore,
Even a single incident of compromised customer data can be costly. most testers will go after the easiest targets first.
Penetration testing helps an organization avoid data incidents that Vulnerability Analysis is the process of identifying vulnerabilities
put its goodwill and reputation at risk. on a network, whereas a Penetration Testing is focused on actually
gaining unauthorized access to the tested systems and using that
G. Justify Security Investments access to the network or data, as directed by the client.
Penetration testing can both evaluate the effectiveness of existing 1. A Vulnerability Analysis provides an overview of the flaws
security products and build the case for proposed Investments. that exist on the system while a Penetration Testing goes on to
provide an impact analysis of the flaws identifies the possible
H. The role of Penetration Testers impact of the flaw on the underlying network, operating
The role of a penetration tester is similar to that of a hacker in system, database etc.
that they are access the information from a network system, but 2. Vulnerability Analysis is more of a passive process. In
their motivation is to improve security. Initially the methods and Vulnerability Analysis we use software tools that analyze
patterns employed by the penetration tester would be similar to both network traffic and systems to identify any exposures
those utilized by hackers. However, penetration testers differ to that increase vulnerability to attacks. Penetration Testing
hackers in that they only probe a network, instead of continuing to is an active practice wherein ethical hackers are employed
exploit and cause malicious damage. A penetration tester is limited to simulate an attack and test the network and systems’
to a specific set of systems they can analyze due to contractual resistance.
obligations. These limitations may take into consideration the 3. Vulnerability Analysis deals with potential risks, whereas
amount of time allocated for the test, which specific systems Penetration Testing is actual proof of concept. Vulnerability
they may probe and the extent to which they may perform the Analysis is just a process of identifying and quantifying the
analysis. security vulnerabilities in a system. Vulnerability analysis
Corporate organizations generally desire the minimum amount doesn’t provide validation of security Vulnerabilities.
of disruption to the functioning of the organization’s main and Validation can be only done by Penetration testing.
back office operations. This means that the process of testing a 4. The scope of a Penetration Testing can vary from a Vulnerability
network and its systems needs to be almost non-intrusive and Analysis to fully exploiting the targets to destructive testing.
that the services the organization provides should continue to Penetration Testing consists of a Vulnerability Analysis, but
it goes one step ahead where in you will be evaluating the et al. suggest without proper and timely assessment, organizations
security of the system by simulating an attack usually done “...often find that their software suffers from systemic faults
by a Malicious Hacker. both at the design level and in the implementation” (Arkin, B.
5. For instance a Vulnerability Analysis exercise might identify et al, 2005). The same can be said for the network security of an
absence of anti-virus software on the system or open ports as organization; without proper and rigorous assessment, the network
a vulnerability. The Penetration Testing will determine the design of an organization will lead to unknown flaws inherent in
level to which existing vulnerabilities can be exploited and the network implementation. There has been limited work on the
the damage that can be inflicted due to this. skills and abilities required of the pen-tester, and less so on the
6. A Vulnerability Analysis answers the question: “What are legal, social, ethical and professional issues arising from such
the present Vulnerabilities and how do we fix them?” A sensitive work. A notable exception to this assertion is the work
Penetration Testing simply answers the questions: “Can any by Pierce, Jones and Warren (Pierce, J. et al, 2007). In their paper
External Attacker or Internal Intruder break-in and what can they provide a conceptual model and taxonomy for penetration
they attain?” testing and professional ethics. They describe how integrity of the
7. A Vulnerability Analysis works to improve security posture professional pen tester may be achieved by “...avoiding conflicts
and develop a more mature, integrated security program, of interest, the provision of false positives and false negatives, and
where as a Penetration Testing is only a snapshot of your finally legally binding testers to their ethical obligations in [their]
security program’s effectiveness. A vulnerability assessment contract” (Pierce, J. et al, 2007). This is certainly noteworthy and
usually includes a mapping of the network and systems should be expected of an individual working with potentially
connected to it, an identification of the services and versions sensitive infor-mation,however this appears mo re of a personal
of services running and the creation of a catalogue of the “ethical code of conduct” rather than something which can be
vulnerable systems. enforced and assessed.
8. A vulnerability assessment normally forms the first part of a Pierce et al. (Pierce, J. et al, 2007) also discuss the then provision
penetration test. The additional step in a by universities “...toward offering security testing courses”.
9. penetration test is the exploitation of any detected Additionally, in 2006, McRue (McRue, A., 2006) commented
vulnerabilities, to confirm their existence, and to determine on the “...first U.K. university to offer a dedicated degree course
the damage that might result due to the vulnerability being in hacking”. This has certainly shown an emerging trend in the
exploited and the resulting impact on the organization. education sector for penetration testing courses, however these
10. In comparison to a penetration test a vulnerability assessment tend to be degree classifications and not necessarily an industry
is not so intrusive and does not always require the same recognized certification standard.
technical capabilities. Unfortunately it may be impossible
to conduct such a thorough assessment that would guarantee A. Requirements of Penetration Testing
that the most damaging vulnerabilities (i.e., high risk) have There are a number of organizational issues that need to be
been identified. addressed before a network penetration test or security review.
11. The difference between a penetration test and a vulnerability These requirements can include legal and contractual issues
assessment is becoming a significant issue in the penetration specifying liability etc. This may also include the technical
testing profession. There are many penetration testers that are requirements involved in the penetration test: The range of IP
only capable of performing vulnerability assessments and addresses over which the test is to be conducted, time constraints,
yet present themselves as penetration testers. If a company the source IP address and the systems that are to be targeted (and
is unfamiliar with the process they may think a networked also those that are not to be targeted) as part of the test. There
system has been fully assessed, when this is not the case. may also be a requirement to inform specific individuals that the
test is taking place. Theoretically there are a number of ethical
J. Commonly Vulnerability Assessment Goes Through and competency issues that penetration testers face in conducting
the Following Phases an assessment, from testing systems or protocols not explicitly
Information Gathering, Port Scanning, Enumeration, Threat included or excluded from a test, to significant omissions that could
Profiling & Risk Identification, Network Level Vulnerability possibly be disastrous to an organization. The penetration tester
Scanning, Application Level Vulnerability Scanning, Mitigation is contractually and ethically bound to abide by the customer’s
Strategies Creation, Report Generation, and Support. Where as requirements, but should ensure the penetration tests is conducted
a Penetration Testing Service however have following phases: correctly and does not lead to a false or misleading sense of
Information Gathering, Port Scanning, Enumeration, Social security.
Engineering, Threat Profiling & Risk Identification, Network Although Code of Conduct and Best Practice is laid out by
Level Vulnerability Assessment, Application Level Vulnerability numerous professional bodies, in actual practice the individual
Assessment, Exploit Research & Development, Exploitation, is often required to take an informed decision given a particular
Privilege Escalation, Engagement Analysis, Mitigation Strategies, situation. Therefore the individual should possess the necessary
Report Generation, and Support. procedural, ethical and technical training.
is because the existing CHECK standard is only applicable for metrics contain vulnerability characteristics which evolve over
Government organizations. Members are provided with guidance the lifetime of vulnerability. Environmental metrics contain those
on standards, methodologies, further recommendations and a code vulnerability characteristics which are tied to an implementation in
of practice. However it should be noted that this information a specific user’s environment. There are different type of attacks
is not publicly available. The scheme provides assurances of are occur according to these attacks which metrics is used and
professionalism to organizations, but not to individuals. we can find the risk.
There are seven base metrics which represent the most fundamental
C. TIGER Scheme features of vulnerability:
The TIGER Scheme (Tiger Scheme, 2010) is focused on providing 1. Access Vector (AV) measures whether the vulnera-
an independent method of determining the skill and ability of bility is exploitable locally or remotely.
a penetration tester. The scheme has a number of levels from 2. Access Complexity (AC) measures the complexity of attack
the Associate membership to the Senior Tester qualification. The required to exploit the vulnerability once an attacker has
structure of the scheme involves separate management committee, access to the target system (high or low).
operating authority and examination body. 3. Authentication (A) measures whether or not an attacker needs
to be authenticated to the target system in order to exploit the
III. International Penetration Testing and Vulnerability vulnerability. (Required or not required)
Assessments 4. Confidentiality Impact (CI) measures the impact on
The introduction of the TIGER Scheme and CREST has shown how confidentiality of a successful exploit of the vulnerability
a governmental initiative has resulted in defining a requirement on the target system. (None, partial or complete)
that industry can follow. When setting up a certification there must 5. Integrity Impact (II) measures the impact on integrity of a
be trusted and experienced professionals that will propose and successful exploit of the vulnerability on the target system.
contribute to the certification standards and these in turn need to (none, partial or complete)
be assessed accordingly. Examination bodies have to be impartial 6. Availability Impact (AI) measures the impact on avail abilty of
and avoid any potential conflict of interest in the accreditation a successful exploit of the vulnerability on the target system.
process and ensure a certain quality is maintained. This can only (none, partial or complete)
be achieved by having an independent examining body with staff 7. Impact Bias (IB) allows a score to convey greater weighting
that has the relevant expertise. to one of three impact metrics over the other two. The value
can be normal (CI, II and AI are all assigned the same weight),
IV. Proposed Work confidentiality (CI is assigned greater weight than II or AI),
A risk means something is about to done or cause harm or reduces integrity (II is assigned greater weight than CI or AI), or
the operational utility of the system. Threats are those things which availability (AI is assigned greater weight than CI or II)
may occur independent of the system under consideration and
which may pose the risk. B. The Temporal Metrics Which Represent the Time
There are two primary methods of risk analysis and one hybrid Dependent Features of the Vulnerability are:
method: Exploitability (E) measures how complex the process is to exploit
• Qualitative - Improve awareness of Information Systems the vulnerability in the target system. The possible values are:
security problems and the posture of the system being unproven, proof of concept, functional, or high. Remediation
analyzed. Level (RL) measures the level of an available solution. (official
• Quantitative - Identification of where security controls should fix, temporary fix, workaround, or un available) Report Confidence
be implemented and the cost envelope within which they (RC) measures the degree of confidence in the existence of the
should be implemented. vulnerability and the credibility of its report. (unconfirmed,
• Hybrid method - A selected combination of these two methods uncorroborated, or confirmed) The environmental metrics
can be used to implement the components utilizing available represent the implementation and environment specific features
information while minimizing the metrics to be collected of the vulnerability. Collateral Damage Potential (CDP) measures
and calculated. It is less numerically intensive (and less the potential for a loss of physical equipment, property damage
expensive) than an in-depth exhaustive analysis. or loss of life or limb. (none, low, medium, or high). Target
• Metrics: IT security metrics can be obtained at different Distribution (TD) measures the relative size of the field of target
levels within an organization. Detailed metrics, collected at systems susceptible to the vulnerability. (none, low, medium, or
the system and network level, can be aggregated and rolled high) Scoring is the process of combining all the metric values
up to progressively higher levels, depending on the size and ac-cording to specific formulas. This is very useful to understand
complexity of an organization. the nature of attacks for pen testers. With the help of these metrics
Good metrics are goal-oriented and should have the following we can find the nature of risk and take the action. The role of pen
features: specific, measurable, comparable, attainable, repeatable, tester is not only achieve the certification but also known the
and time dependent. behavior and nature of different types of vulnerabilities. So the
role of pen tester is very important in the organization. There are
A. Metrics to Evaluate the Security Vulnerabilities different tools are used for pen test:
A Common Vulnerability Scoring System (CVSS) which was
designed to calculate the risk of a vulnerability. The score is C. Port Scanners
derived from metrics and formulas. The metrics are in three distinct Port scanning tools are used to gather information about a test target
categories that can be quantitatively or qualitatively measured. Base from a remote network location. Specifically, port scanners attempt
metrics contain qualities that are intrinsic to any given vulnerability to locate which network services are available for connection on
that do not change over time or in different environments. Temporal each target host. They do this by probing each of the designated
(or default) network ports or services on the target system. Most References
port scanners are able to scan both TCP as well as UDP ports. [1] IDC, (2009),“Number of Mobile Devices Accessing the
Most can also target a specified list of ports and can be configured Internet Expected to Surpass One Billion by 2013”, Reported
for the speed and port sequence that they scan. on 9 Dec 2009, [Online] Available:// http://www.idc.com/
getdoc.jsp?containerId=prUS22110509.
D. Vulnerability Scanners [2] Moses A., (2010),“Internet addresses running out”, Sydney
The primary distinction between a port scanner and a network- Morning Herald, [Online] Available:// http://www.stuff.
based vulnerability scanner is that vulnerability scanners attempt co.nz/dominionpost/national/technology/3958727/Internet-
to exercise (known) vulnerabilities on their targeted systems, addresses-running-out
whereas port scanners only produce an inventory of available [3] ACPO, (2009),“ACPO e-Crime Strategy 2009 Report: A
services. That said, the distinguishing factors between port and Strategic Approach to National e-Crime”.
vulnerability scanners are often times blurred. Apart from that, a [4] Markkoff, J. (2008). Before the Gunfire Cyberattacks.
good vulnerability scanner is a vital tool to a traditional penetration New York Times. [Online] Available: http://www.nytimes.
tester. They provide an essential means of meticulously probing com/2008/08/13/technology/13cyber.html?_r=1. 2
each and every available network service on the targeted hosts. [5] Higgins, K. J. (2010),“Anatomy Of A Targeted, Persistent
Vulnerability scanners work from a database of documented Attack”, DarkReading, 27 Jan. 2010, [Online] Available:
network service security defects, exercising each defect on each http://www.darkreading.com/database_security/security/
available service of the target range of hosts. attacks/showArticle.jhtml?articleID=222600139
[6] Dekker, M. (1997),“Security of the Internet”, CERT
E. Application Scanners Coordination Center Reports, [Online] Available: http://
Taking the concept of network-based vulnerability scanner one www.cert.org/encyc_article/tocencyc.html
step further, application scanners began appearing several years [7] Stoll, C. (1989),“The Cuckoo’s Egg: Tracking a Spy Through
ago. These attempts to do probing of general purpose web-based the Maze of Computer Espionage”, Doubleday, NY, USA
applications by attempting a variety of common and known attacks [8] EC-Council, (2010). Certified Ethical Hacking Training
on each targeted application and page of each application. Course. [Online] Available: http://www.eccouncil.org/
certification/certified_ethical_hacker.aspx
F. Web Application Assessment Proxy [9] Bentley, L., (2006), “Penetration Testing Key to HIPAA
Although they only work on web applications, web application Compliance for Care New England”, IT Business Edge,
assessment proxies are perhaps the most useful of the vulnerability [Online] Available: http://www.itbusinessedge.com/cm/
assessment tools listed here. Assessment proxies work by community/features/interviews/blog/penetration-testingkey-
interposing themselves between the tester’s web browser and to-hipaa-compliance-for-care-new-england/?cs=22127
the target web server. Further, they allow the tester to view and [10] Cabinet Office, (2009), “Cyber Security Strategy of the
manipulate any and all data content flowing between the two. United Kingdom”, June 2009, [Online] Available: http://
This gives the tester a great deal of flexibility in trying different www.cabinetoffice.gov.uk/media/216620/css0906.pdf
“tricks” to exercise application weaknesses in the application’s [11] Arkin, B., Stender, S., McGraw, G.,“Software Penetration
user interface and associated components. This level of flexibility Testing”, IEEE Security and Privacy, Vol. 3, Issue 1, 2005.
is why assessment proxies are considered essential tools for all [12] Pierce, J., Jones, A., Warren, M.,“Penetration Testing
black box testing of web applications. Professional Ethics: a conceptual model and taxonomy”,
Australasian Journal Of Information Systems, 13(2). [Online]
G. Important Feature for Selecting the Right Toolkit Available: http://www.dl.acs.org.au/index.php/ajis/article/
Following are the features for right toolkit: view/52
Visibility, Extensibility, Documentation, License flexibility. [13] McRue, A. (2006),“University opens school for hackers”.
[Online] Available: http://news.cnet.com/University- opens-
V. Conclusion schoolfor- hackers/2100-7355_3-6085375.html
Although penetration testing is an industry recognized term, there [14] IEEE, (2010),“The Institute of Electrical and Electronics
is still ambiguity as to what a penetration tester actually does and Engineers”, [Online] Available: http://www.ieee.org/
how they provide assurance that the work they carried out is fit for [15] BCS,“BCS - The Chartered Institute for IT”, (2010), [Online]
purpose. It is important to make a distinction between penetration Available: http://www.bcs.org
testing and network security assessments. [16] IEEE, (2010),“IEEE Computer Society”, [Online] Available:
A network security or vulnerability assessment may be useful to a http://www.computer.org
degree, but do not always reflect the extent to which hackers will [17] BCS, (2010),“BCS Information Security Specialist Group
go to exploit a vulnerability. Penetration tests attempt to emulate (BCS-ISSG)”, [Online] Available: http://www.bcsissg.org.
a ‘real world’ attack to certain degree. The penetration testers uk
will generally compromise a system with vulnerabilities that they [18] The Institute of Information Security Professionals (iisp),
successfully exploited. Hackers and intruders need to find only one (2010) [Online] Available: https://www.instisp.org/SSLPage.
hole to exploit whereas penetration testers need to possibly find aspx?pid=183
all if not as many as possible holes that exist. This is a daunting [19] The ISC2 Code of Ethics, [Online] Available: https://www.
task as penetration tests are normally done in certain time frame. isc2.org/ethics/default.aspx
Finally, a penetration test alone provides no improvement in the [20] Council of Registered Ethical Security Testers (CREST),
security of a computer or network. Action to taken to address (2010). [Online] Available: http://www.crestapproved.org/
these vulnerabilities that is found as a result of conducting the Pages/RequiredMembership.html
penetration test.