White Box Or Black Box For Social Engineering Testing?

When conducting social engineering testing as part of an audit or security assessment, should the client provide a listing of employees to test? Doing so is generally termed white box testing, as detailed information is provided to the auditor. The term “white box testing” was originally used to describe a form of software testing where detailed information on the software application was provided to the person reviewing the code. The same principle can be applied to other areas of review, such as social engineering testing. Conversely, black box testing is when very little information is provided to the auditor. Again, the concept is derived from software testing. In black box testing, the auditor attempts to obtain information using various sources and tools in order to find and exploit weaknesses.

From an auditor’s perspective, determining which method to use depends on what the client is trying to accomplish. If the audit is to test the client’s employees’ susceptibility to social engineering, then having the client provide a list of employees and contact information streamlines the process and makes the testing more comprehensive. Given enough time and effort, an attacker will obtain detailed information on a company. Having much of this information provided by the client minimizes the billable time on the project and still provides the desired results: identifying which employees respond to social engineering tactics. More of the auditor’s time can be spent on testing vs. reconnaissance activities.

Conversely, if the intent of the audit or test is to evaluate the client’s ability to obscure information, then the test should require the auditor to obtain information through tools and techniques without the client providing the information freely. However, this process significantly increases the cost and might not find the employees that are most susceptible to social engineering tactics. Remember, an attacker only needs to find the right person to have a successful attack. With a black box test, the auditor may not find all of a company’s employees. Consequently, they may not be able to conduct attacks on very many people, which may inappropriately give a false sense of security. Just because a black box test does not find a susceptible person, does not mean that additional time and effort could not produce a different result. An audit or security firm will have budgetary constraints that may not be present for a true attacker. The only payoff for the audit firm is the contracted fee. On the other hand, an attacker may spend weeks, months, or years conducting an attack as the payoff may be much larger, from fraudulent transactions to theft of intellectual property.

Deciding between white box testing and black box testing really depends on the end goal of the test. If the intent is to test employees’ susceptibility to social engineering, white box testing may be more effective. On the other hand, if the intent is to determine a company’s ability to obscure or not disclose information, then black box testing may be more appropriate. As attackers do not necessarily have time or budget restrictions, they can spend more time on an attack, whereas a contracted testing firm is in business and their only payoff is the fee for the engagement. Although black box is more realistic in its purest form, white box testing will generally provide better results.