Pen Testing For Web Applications: Planning and Processes

Planning a penetration test for a web application can require some specific knowledge regarding the security testing process, such as:

• The types of testing methods that are available for web applications
• The differences between manual testing techniques and automated tools
• How much and how long can a web application test take to complete
• What type of vulnerabilities may be identified
• Is there any direct risk for your company, without conducting a security test

The following article aims to provide some useful information regarding web application penetration testing and to help your business plan out your next security test.

Manual Testing And Automated Tools

There are a large number of security testing tools that advertise automated scanning and vulnerability identification for web applications.

Many scanning tools also advertise that they can provide automated pen testing for web applications, as well as additional features such as network penetration testing, and internal penetration testing.

With such a range of automated tools available, which can often provide a regular scanning service at a lower price than hiring a security consultant, a common question can often be why conduct manual web application penetration testing.

The following article highlights some of the differences between these two methods and how to balance the strengths and weaknesses of the two to provide an effective security assessment of your applications.

What Is Involved With Web Application Penetration Testing

The web application penetration testing process will follow a similar overall structure as the penetration testing phases described in the following article, which includes:

• Scoping, to determine what is to be included or excluded from the assessment
• Reconnaissance, to discover accessible content, functionality, and services that can be assessed
• Vulnerability testing, to identify vulnerabilities through a combination of automated and manual testing techniques
• Reporting, to define and describe the findings of the assessment and provide useful mitigation and remediation information

For any penetration test that is conducted, your outcome should include a detailed list of vulnerability findings and remediation methods to address these findings.

Depending on the testing company, the specific contents and format of the report can vary and could be provided through its own web portal, or through a PDF or similar documentation.

How Long Will A Web Application Penetration Test Take

Web application penetration testing doesn’t have any specific time associated with it. Due to the large variety within different web applications, the amount of time required for testing can also vary.

Some applications may be relatively simple and only consist of a few pages, in which case a single day may be more than enough time to conduct an assessment.

However, other web apps can consist of a large amount of content, with varying and diverse features, multiple authentication systems, and permission levels. For applications such as this, it may take several days just to conduct sufficient reconnaissance of all of its content before a cyber security assessment begins.

Whether conducting a manual test or using automated tools, this variation in time to complete a security assessment can still occur to some degree.

For some companies, there may be strict budgets that limit the amount of time that can be spent on a security assessment of your web application.

Depending on the testing companies, some may agree to limit the total amount of time they spend on testing your application to align with your budget, however, this will likely mean that areas of your application have not been sufficiently tested, and vulnerabilities may remain unidentified.

How Much Does A Web Application Penetration Test Cost

The cost of any web app penetration test will inevitably be tied to the time that is spent on the testing process.

With different testing companies in the UK, a guide on price can sometimes be estimated at around £1000 per day for web app pen testing, however, this can often vary depending on the company and the security testers’ individual experience and qualifications.

Vulnerability scanning tools may often advertise that they provide automated penetration testing services.

While scanning tools can reduce the cost of testing, and be carried out on an automated schedule, there are limits to their effectiveness, and they are typically not as thorough as investing in a manual testing process.

Web Application Vulnerability Testing Techniques

There are a variety of methods that can be used to identify vulnerabilities within web applications.

Security testing can be incorporated into the development process or can be applied to applications that are already setup and running.

Ideally, vulnerability testing can be considered as a layered and multi-staged approach, being conducted at all stages of the application development process.

However, this can also be a costly process, and not all organizations may maintain the security budget to incorporate such extensive testing.

Where budgetary restrictions are faced, this can justify a blend of both automated and manual testing techniques, to conduct as much security testing as possible while keeping costs down.

The following are examples of security testing methods that are often carried out, which can be conducted using manual or automated techniques.

Static Application Security Testing (SAST)

Within the development process functionality testing is conducted to ensure the developed content is working as intended.

As part of the software development lifecycle, security testing can also be conducted against the source code, as different functionality and features are developed.

Incorporating security testing into this process can review the developed code and features of the web application to ensure that new functionality is developed in a secure manner.

SAST is often conducted using automated tools and a range of scanning tools advertise this type of security testing functionality.

Dynamic Application Security Testing (DAST)

Dynamic security testing focuses on the web application while it is currently running and interacting with back-end authenticated systems, databases, and other technologies.

Rather than a review of the underlying source-code, dynamic testing focuses on the working functionality and how the variety of requests submitted to the application can introduce vulnerabilities.

Many automated and manual testing processes focus primarily on this type of security testing approach, and there are also a range of approaches and terms that can further categorize the different types of DAST testing techniques.

Unauthenticated Web Application Penetration Testing

Where your web app has user authentication features, pen testing may be conducted with or without authentication.

An unauthenticated web application test can allow your business to gain insight into the security weaknesses that are likely to be targeted by unknown external attackers who have no prior access to your application.

While this can be useful insight into the security of your web application, it can also limit the knowledge you may gain access to, regarding the overall security of your application and all of the security issues which your web app may be affected by.

Authenticated Web Application Penetration Testing

Authenticated testing can account for the weaknesses of unauthenticated testing by providing authentication information to the penetration testers.

This allows additional features and functionality within your application to be assessed, which may highlight a range of vulnerabilities that would otherwise be overlooked by unauthorized users.

This can provide greater context into the security of your applications, and help your business determine the potential security risks, in the event an account is compromised, through weak or reused passwords, or through attacks such as Phishing which lead to a compromised account.

Source-Code Assisted Penetration Testing

Rather than using an automated tool to assess possible vulnerabilities within the source-code of your application, code-assisted pen testing can be conducted to evaluate how a user may interact with your web app and the underlying code behind each function and feature that may be interacted with.

This is typically the most exhaustive type of web application penetration test, and can therefore require a longer period of time to complete and also be more expensive as a result.

Black Box Testing For Web Applications

Similar to unauthenticated testing, Black Box testing is another term for providing penetration testers with little or no information regarding your web application, and determining the security risks that may impact your application from the perspective of cyber attacks with no prior information or access.

Gray Box Testing For Web Applications

Gray box testing is a bridge between black box testing and white box testing, or unauthenticated testing and source-code assisted penetration testing.

With a gray box test, some information is provided to a security tester, such as information regarding the technologies in use for your application and several test user accounts, however, detailed knowledge and privileged accounts are still withheld.

This can provide insight into the security of your application should a standard user account be compromised, through a range of account compromise techniques.

White Box Testing For Web Applications

Similar to source-code assisted penetration testing, a White Box test is another term that can refer to a penetration test where all available information is provided to the security testers to conduct a more in-depth security audit and test of your application.

Although this type of testing process can often be longer and more expensive to carry out, it can also provide the most exhaustive review and detailed security assessment of your web application.

Why Conduct Web Application Penetration Testing Services

The question of “why conduct web application penetration testing” is often raised by many businesses, which have not had any direct experience with cybersecurity incidents, or may not consider the security risks that they may face from different online cyber-attacks.

Unfortunately, web applications can be impacted by a large number of vulnerabilities, with many vulnerabilities that are unique to web applications.

Billions of potential attacks are launched against web applications each year, and new vulnerabilities are continually identified and exploited within web applications.

As cyber-attacks are largely untargeted, looking for specific vulnerabilities in any responsive web applications, the size and type of business are often ignored, as automated scanning and exploit tools are designed to compromise any systems they can access.

Why Web Applications Are Targeted For Attack

After exploitation, an attacker may use their established access, for a variety of reasons, such as:

• Targeting additional systems within a business to which a web application may be connected to
• Targeting a company’s clients with Phishing attacks through information collected from the web application
• Embedding additional code into the application to harvest user information, credentials, or credit card data
• Utilizing the web application server as a platform to launch other attacks against other businesses
• Incorporating the compromised web application into a network of compromised systems used for Distributed Denial of Service attacks

Cyber-attacks often aim to exploit vulnerabilities and do not target a specific business. The result of identified vulnerabilities isn’t necessarily to access sensitive information but can be seen as a method to access other businesses.

Adhering to security best practices is therefore recommended, in addition to ensuring your business will test web applications for cyber security flaws to mitigate the risks of data breaches.

What Are Some Common Vulnerabilities In Web Applications

Web applications can be impacted by many unique security vulnerabilities, some of which may be considered of Critical Impact while others may be considered Low Impact or simply informative recommendations for best practice security.

While some automated scanning tools can be effective at identifying web application vulnerabilities there can often be a number of security flaws that will not be identified without manual web application penetration testing techniques.

Some of the vulnerabilities that are often included within the OWASP Top Ten and can frequently impact web applications include:

Cross-Site Scripting (XSS)

Cross-site scripting involves an attacker embedding their own custom Javascript into the responses that are returned by the web application.

This allows for attacker-customized functions to be defined, which may be intended to steal user information or present alternative forms, such as login prompts to the user.

There are several variations of an XSS attack, and each requires a user to navigate to an affected page within the web application, often using a URL that has been provided by an attacker.

SQL Injection (SQLi)

SQL Injection allows an attacker to interact with the web application’s back-end database, through customized requests submitted to the application.

This process can allow an attacker to request information from the database, such as user data, credentials, and other information stored within the database.

Depending upon the type of SQLi, the attacker may also be able to add content to the database or compromise the operating system of the database server.

Insecure Direct Object Reference (IDOR)

IDOR allows an attacker to request information or utilize functionality that is not intended to be accessible to them, as the access control that restricts these features has not been correctly implemented.

This may result in the account information of other users being requested or could allow administrative functions to be executed using a low-permissioned account.

Insecure Authentication Systems

An insecure authentication system could be implemented in several different ways, which allows an attacker to manipulate or compromise the login process for an application.

• Username Enumeration, allows an attacker to identify valid usernames for the application through monitoring varying responses when supplying a valid or invalid username.
• Insecure Password Reset could allow an attacker to alter the password of a legitimate user by manipulating the password reset process which many web applications feature a version of.
• No Brute Force Protection means that an attacker may submit thousands of passwords to the application, with a known valid username, without ever facing restrictions in their login attempts.

Limitations In Automated Scanning Tools

Automated testing tools, while efficient at running vulnerability tests, can still be found to have gaps in the overall number and types of vulnerabilities that they are able to identify.

While the number of web application security vulnerabilities that scanning tools cannot find is continually shrinking, there still remain certain types of vulnerabilities that automated tools can struggle to identify, such as IDOR.

Insecure Direct Object Reference

Insecure Direct Object Reference (IDOR) is a common web application vulnerability that results from insecure access controls being implemented within an application. The vulnerability can often result in a user being able to access sensitive data and the data of other user accounts.

It can also be overlooked by many automated web penetration testing tools as its identification can often require a response from the web application being reviewed and its contents contextualized with regard to a user’s current authentication status.

Identifying IDOR

As an example of IDOR, if your web application allows you to login and view your account information by accessing a URL such as https://example.website.com/accounts?account=12345

This may present your user information, such as full name, address, login details, and other personal details.

However, if you directly alter the listed URL, and change the account number to, account=12344, this may change the information displayed on your account page to the account information for a different user.

This outlines a principle of an IDOR security issue, however, in many instances, the specific method to identify such a vulnerability has more complexity than this, which can often be why a vulnerability scanning tool is unable to identify the issue.

Impacts of IDOR

In many other cases, the issue is not restricted to just viewing the information of another user, but also extends to changing the information of another user.

This can also extend to accessing functionality which should only be accessible by an administrator account. This process can allow your own account permissions to be altered, to raise your account status to that of an administrator.

Automated web application pen testing tools, although continually improving and increasing their capability to identify vulnerabilities, still have many limitations, which can require a manual testing process to provide broader coverage to account for these limits.

Limitations In Manual Testing For Web Applications

Limitations in automated tools does not mean that a security consultant is flawless in their approach and incapable of overlooking vulnerabilities.

While scanning tools can encounter issues with how they are able to detect vulnerabilities, security testers are still fallible and capable of human error when conducting penetration testing for web applications.

Both manual testing and automated scanning tools have their place within a vulnerability management program, and the strengths of each should be used to your advantage, to account for and minimize the weaknesses that can be associated with each security testing process.

• Within a robust vulnerability management program, manual testing can often be utilized as an initial security assessment, to identify as many known vulnerabilities as possible.
• This can be followed with regular vulnerability scanning, to account for any additional or new vulnerabilities which may be identified over time.
• Further manual testing can then be reviewed annually or upon any major development work, to ensure that no overlooked issues are introduced into your application.

Benefits Of Web Application Scanning Tools

The primary benefits of using scanning tools for web app penetration testing are their speed and cost.

A vulnerability scan can often be setup, launched, and concluded within the same day, and regular scans can be scheduled to commence every day, week, or month, depending on your specific requirements.

Benefits Of A Manual Test For Web Applications

A manual web app pen test is a slower process and can require more planning and scheduling.

However, web application penetration testing will typically be more in-depth, with a broader range of security flaws identified, and a greater context applied to the findings and how they relate to your business.

Working With A Penetration Testing Partner

Web application penetration testing companies are not all identical and there can be differences in their approach to planning, testing, and delivering a report.

Different companies and individuals may also have different penetration testing certifications, which may demonstrate their knowledge, experience, and expertise in delivering web application penetration testing.

When working with or choosing your provider, it can be useful to gather some information from each potential supplier to allow your business to make a more informed decision, such as:

• What certifications does the business maintain for information security, quality assurance, and penetration testing
• What certifications does the individual penetration tester hold who will be conducting the assessment
• What is the documented process and methodology that is followed for conducting a penetration test
• What are the contact and communication methods in place, before, during, and after a penetration test
• What is the reporting structure that is provided and does it include information such as:
  • A Vulnerability description to define what the target system is affected by
  • Vulnerability remediation efforts, detailing how a business can resolve security vulnerabilities
  • Vulnerability Risk Rating or use of the Common Vulnerability Scoring System (CVSS) defining the security risks of the issue
  • Executive High-Level Summary of Risks and Security Posture, outlining the contextualized business risks and improved security measures that can be put in place

Additionally, a major determining factor for any business will often be price, and as different penetration testing companies will have different rates they apply, acquiring an estimated cost or quote for your required test can be useful when making your final decisions.

Conclusion

The web applications in use for most businesses are continually targeted by attackers and face a number of cyber threats aiming to exploit identified vulnerabilities and gain access to sensitive data.

Web application pen testing can help mitigate such threats and improve web application security, through the systematic process of identifying vulnerabilities which can determine additional security measures and security policies to enact for your business.

Some of the most common factors that restrict a business from conducting web app penetration testing services include a lack of security awareness of the risks that a business may face and the overall cost of conducting a web application penetration test.

The cost of web app penetration testing can ideally be mitigated by using a combination of automated web app pen testing tools and consulting with experienced web application penetration testers to ensure your company can address security weaknesses within your applications.

For further information on developing processes within your business to address vulnerabilities the following articles can be useful:

• Conducting a Security Risk Assessment
• Developing a Vulnerability Management Process
• Prioritising Vulnerabilities Within Your Business
• Implementing a Vulnerability Scanning Tool
• Security Solutions For Small Businesses

Alternatively, if your business is considering cyber security vulnerabilities for your devices and systems as a whole, it can be useful to develop policies and practices to adhere to a regulatory compliance standard.

Cyber Essentials and Cyber Essentials Plus, can be implemented within your business which can incorporate testing for security vulnerabilities within your devices, network, and web apps.

Although the type of testing within such a compliance standard is not as extensive as penetration tests, it can be useful to provide broad coverage of your business, while also remaining affordable for most companies.

Where you have any further questions regarding different cybersecurity solutions or the Cyber Essentials certification program, our consultants are available to address any concerns you may have.