The limitations of penetration testing
Although penetration tests are recommended and should be conducted on a regular basis,
there are certain limitations to penetration testing. The quality of the test and its results will
directly depend on the skills of the testing team. Penetration tests cannot find all of the
vulnerabilities due to the limitation of scope, limitation of access of penetration testers to
the testing environment, and limitations of tools used by the tester. The following are some
of the limitations of a penetration test:
Limitation of skills:
As mentioned earlier, the success and quality of the test willdirectly depend on the skills and experience of the penetration testing team.
Penetration tests can be classified into three broad categories: network, system,
and web application penetration testing. You will not get correct results if you
make a person skilled in network penetration testing work on a project that
involves testing a web application. With the huge number of technologies
deployed on the internet today, it is hard to find a person skillful in all three. A
tester may have in-depth knowledge of Apache web servers, but might be
encountering an IIS server for the first time. Past experience also plays a
significant role in the success of the test; mapping a low-risk vulnerability to a
system that has a high level of threat is a skill that is only acquired through
experience.
Limitation of time:
Penetration testing is often a short-term project that has to be completed in a predefined time period. The testing team is required to produce
results and identify vulnerabilities within that period. Attackers, on the other
hand, have much more time to work on their attacks and can plan them carefully.
Penetration testers also have to produce a report at the end of the test, describing
the methodology, vulnerabilities identified, and an executive summary.
Screenshots have to be taken at regular intervals, which are then added to the
report. Clearly, an attacker will not be writing any reports and can therefore
dedicate more time to the actual attack.
Limitation of custom exploits:
In some highly secure environments, normal
penetration testing frameworks and tools are of little use and the team is required
to think outside of the box, such as by creating a custom exploit and manually
writing scripts to reach the target. Creating exploits is extremely time consuming,
and it affects the overall budget and time for the test. In any case, writing custom
exploits should be part of the portfolio of any self-respecting penetration tester.
Avoiding DoS attack:
Hacking and penetration testing is the art of making a
computer or application do things that it was not designed to do. Thus, at times, a
test may lead to a DoS attack rather than gaining access to the system. Many
testers do not run such tests in order to avoid inadvertently causing downtime on
the system. Since systems are not tested for DoS attacks, they are more prone to
attacks by script kiddies, who are just out there looking for such internet-
accessible systems in order to seek fame by taking them offline. Script kiddies
are unskilled individuals who exploit easy-to-find and well-known weaknesses
in computer systems in order to gain notoriety without understanding, or caring
about, the potential harmful consequences. Educating the client about the pros
and cons of a DoS test should be done, as this will help them to make the right
decision.
Limitation of access
Networks are divided into different segments, and the
testing team will often have access and rights to test only those segments that
have servers and are accessible from the internet in order to simulate a real-world
attack. However, such a test will not detect configuration issues and
vulnerabilities on the internal network where the clients are located.
Limitations of tools used:
Sometimes, the penetration testing team is only
allowed to use a client-approved list of tools and exploitation frameworks. No
one tool is complete irrespective of it being a free version or a commercial one.
The testing team needs to be knowledgeable about these tools, and they will have
to find alternatives when features are missing from them.
In order to overcome these limitations, large organizations have a dedicated penetration
testing team that researches new vulnerabilities and performs tests regularly. Other
organizations perform regular configuration reviews in addition to penetration tests.