Ask a Pen Tester: Q&A with Rapid7 Penetration Tester Aaron Herndon

Last updated at Thu, 01 Feb 2024 20:45:23 GMT

Recently, we gave our customers the opportunity to ask members of our penetration testing services team any burning questions they have. They did not hold back, and neither did I. I hope you learn something from my shared tactics, experiences, and perspectives. Enjoy!

Which tool do you use to collect the evidence you found?

If we are referring to evidence as exfiltrating sensitive data from a compromised machine to test data loss prevention capabilities, it is highly scenario-dependent. Unless the client specifies otherwise, we place simulated sensitive data on the compromised host, then exfiltrate that. An example of this is when we compromise a server with credit card data. It would be unethical to exfiltrate legitimate consumer credit card information, so instead, we have written scripts that generate fake credit card data on the compromised server so we exfiltrate only the mock data.

To exfiltrate the data, I like using tools that communicate over HTTPS, though we often have to assess our position in the network to determine what legitimate traffic looks like, then attempt to impersonate it. Therefore, I cannot say there is a singular tool or method I often use for exfiltration of data. However, one thing that remains constant across penetration tests is the need to capture screenshots as proof for writing reports. For this, I like to use the screenshot tool Greenshot, since it makes annotations and obfuscations a snap.

Do you use any of the Hak5 gear when out in the field?

I most definitely do. My personal favorite is the LAN Turtle (so many applications). I intend to buy a BashBunny at some point, as I have been wanting to try it out. I have used my LAN Turtle many times as a plant on a network after physical intrusion to establish a reverse socks proxy via SSH to my server on the internet. It might not be blazing fast, but it is compact, gets the job done, and looks a little less suspicious than my rigged-up Raspberry Pi.

What is your favourite and most common exploit you use?

While this is more of a misconfiguration abuse than an exploit, I like extracting cleartext credentials cached in memory, most commonly within the LSASS process on Windows systems yields promising results. To date, it is the easiest way to obtain credentials for lateral movement or elevate privileges within a network.

How prevalent is NetBios and LLMNR out in the field?

LLMNR and NBT-NS poisoning is near the top of the list in the quick-win network compromise playbook. Unfortunately, it’s very common on internal network penetration tests and often leads to the initial foothold inside of a network. We often see clients remediate the issue via their configuration management tool of choice, only to discover on our next test that a handful of assets did not receive the configuration change. In our penetration test reports, we list the tools and the exact command-line arguments used to achieve our testing results, aiding clients in their ability to validate patches and configuration changes by replicating our steps.

Have you ever been in a situation where you had a USB drive connected to a device and watched the percentage copied data counter creep slowly toward 100% while someone is coming toward you? Or is it just easier now to transfer the data wirelessly to another device?

Yes! In true Hollywood hacker fashion, this happened to me in a casino. I was booting to a Linux operating system on my thumb drive to quickly grab the SAM and SYSTEM files from an unencrypted workstation. I had tailgated my way into the employee “back of house,” made a few turns down long hallways, and at the end of one, I discovered an employee training computer lab. It was midday and the lights were off with no one inside, so I stepped in, sat down, and went to work.

After booting to the USB, I glanced up to quickly check if anyone was nearby. Out of the small, horizontal window in the door, I could see the security guard walking down the hall toward the room (the same security guard who had checked my ID 20 minutes prior when I entered the casino as a patron). With my heart pounding, I quickly pulled up a terminal, mounted the Windows hard drive, and began copying the SAM and SYSTEM files to a USB drive. While the files are small and take an unnoticeable amount of time to copy, the process to mount the drive and copy the files is a few commands. The hallway was also the only available exit.

Once I had the files, I quickly flipped the power switch on the desktop and pulled out my phone, pretending to be on a phone call. The guard opened the door and gave me a quick look. I briefly waved my hand to acknowledge his presence, but I maintained a sideways stance to him and did not lift my head to look him in the eyes. Continuing my fake phone conversation, I began yelling into the phone as if I were in a heated conversation. People do not like to stop others when they are on the phone, especially when they appear to be stressed. Out of the corner of my eye, I could see the guard wave at me then turn and walk away. Whew. It was just standard patrol rounds of the back of house, and he did not recognize me from our earlier encounter. I had my files. Now it was time to extract the NTLM hash, crack it, and return with the local admin password at a later time.

What's the quickest time you've gotten Domain Admin?

On an internal network penetration test, without any network port security to bypass, seven minutes. It was the classic sit down, run Responder and Ntlmrelayx, Domain Admins were logged into their workstations on the same subnet (lack of principle of least privilege), and a simple relay of their coerced authentication to my machine (via Responder) to a file server. One and done.

What is more effective in your experience when composing phishing email—being aggressively demanding or referencing others at the company as having directed you to the person?

This depends on your pretext. If you are able to impersonate an employee's superior, being demanding has some merit. However, I have often found that when you are demanding, people take a defensive posture. Once a person is on the defense, it will be hard to coerce them into conducting your proposed actions.

Personally, I prefer to take the business-professional approach, meaning that my phishing emails are void of spelling and grammar mistakes, make use of polite language, and are concise with their request. A common tactic is to impersonate a third-party platform that the target company's "administrator" has invited employees to use. The email can be sent by a generic account from the mock third-party site, with a subdomain registered to the target company (i.e., target.acmetraining.com) and the body indicating that their account has been enrolled with Single-Sign-On on the target company's "new third party portal."

Fountain or ballpoint pens?

Good one! My latest research conducted against fountain pens resulted in the discovery of a critical vulnerability dubbed #InkBleed. Those are my favorite. Though the floating point exception issues in ballpoint pens are still interesting. #BadDadJokeAttempt

In your opinion, which makes your job harder (aka makes you easier to stop or detect): host-based tools or network-based detection tools?

From an external network perspective, when I am attacking a client's externally facing assets, network-based detection and response tools are often the most effective. Specifically, an external network that is well monitored against port scanning, password spraying, web service scans, etc., and is configured to automatically block the offending IP slows down the process significantly.

When it comes to phishing with payloads to get access or moving laterally within an internal network, host-based detection and response tools are the most effective. Both have their merit and are a necessity for a comprehensive security program. My primary passion is developing payloads to bypass host-based protections, so from that perspective, host-based tools often give me the most trouble because I am seeking them out and spending a considerable amount of time to develop those bypasses. Additionally, a host that is well monitored via an endpoint detection and response (EDR) solution can unravel a red team operation. One mistake that the EDR flags during your reconnaissance or exploitation can result in your foothold being lost, if the security team properly responds to the alert.

What do you see as the true value of a penetration test?

When done right, penetration tests offer insight into a company's network and/or application that automated tools, such as a vulnerability scanner or application scanner, may miss. Often when we test clients with a mature security posture, we are not expecting to find common vulnerabilities such as MS17-010. A vulnerability scanner likely picked those up, and the company patched. Instead, we are hunting for misconfigurations in the network that may allow a foothold, or enough information to be extrapolated into a foothold. On top of this, penetration tests are a collaborative process with the client. We make every effort to be transparent with our clients, and offer an opportunity for their team to ask questions and learn along the way.

Overall, penetration tests allow companies to see beyond the vulnerabilities presented by a scanner, and get a holistic picture of the vulnerabilities that lie within their network. For clients that are still getting a handle on what to patch after receiving a vulnerability scan report, a penetration test can aid in understanding what security measures should be prioritized and the impact of failing to remediate the issues. Beyond a penetration test, once companies are ready to assess their ability to detect and respond to realistic targeted threats, there is the Rapid7 Red Team service offering. These are personally my favorite, as they yield the best insight into a realistic scenario, and the collaborative debriefs at the end of the assessment help both the red and blue teams learn.

Do you usually have better results with onsite or remote pen tests?

Assuming that this question is referring to internal network penetration tests, I would personally say that onsite assessments yield better results. However, this is not in terms of the quality of findings, but the overall quality of the engagement. I will find the same vulnerabilities whether I am remote or onsite.

However, when we’re onsite, the in-person interaction with the point of contact allows for our findings to be better articulated as we discover them. We often end up sitting in a room with the point of contact, demonstrating our attacks against their network, and helping them understand our methodology more than we could have if we were remote.

Do you see more benefit in traditional pen testing or red/blue team scenarios?

It depends on the maturity level of the client. For a company without a dedicated blue team, or one that has not yet deployed any security monitoring or alerting tools like a SIEM, I would recommend a penetration test. For a client that has a moderately mature security posture, especially those with their own Security Operations Center (SOC), I would recommend red/blue scenarios.

However, I feel that both tests can complement each other. A network penetration test can be a precursor for the red team and provide a holistic view of vulnerabilities on the network. The penetration test offers more time to the tester to identify every vulnerability they can, without having to factor in the reduction in speed and tactics due to stealth. After the penetration test has completed, and the company has had adequate time to remediate vulnerabilities identified, a red team assessment can be conducted as an exercise to sharpen the company's blue team and test the effectiveness of detection and response tooling.

Do you believe internal sanctioned penetration tests are more valuable focusing on automated CI/CD pipelines or the containers/microservices they create and provision?

This depends on the risk a client is looking to assess. Penetration tests are often conducted in a black-box fashion, without prior knowledge of the underlying infrastructure and application code. From an application perspective, the idea is to stress the application beyond tests that may have been conducted in the CI/CD pipelines, looking for code that may be "bug-free" according to build tests, but may have oversights in application logic that can be exploited.

There is merit to this style of test, as it is what legitimate malicious actors will face when they approach the application to attack it. However, while not a penetration test, a code review or design review of the tests run during the CI/CD pipeline may help discover additional flaws that could not be uncovered from a black box perspective.

Personally, my favorite application security test is one in which the client provides both a test environment of the application and the ability to review the code, tests, and anything that may have been conducted during the CI/CD pipeline to look for oversights. Putting both together provides a full picture to the tester and allows them to validate any suspicions they may have about potentially vulnerable areas of code.

What's your opinion regarding the hack back controversy? It appears it's getting more traction, and I think there are pros and cons to this but could have a lot of implications.

I am against it for the private sector. Personally, I feel that innocent companies will be caught in the crossfire, when their compromised hosts are being used by a malicious actor to proxy traffic, and they now fall in the crosshairs of the "ethical hacker" trying to catch the “bad guy." The amount of policies and procedures that would need to be in place to make hack back reasonable is something I do not feel can be adequately policed in the private sector.

What are the top 5 vulnerabilities or misconfigurations you find in most engagements?

I actually gave a talk about this a couple of years ago at GrrCon titled, "Pen Test War Stories: Easy Ways to Make My Job Harder." As a summarized response, the easy-win list on a network penetration test is as follows:

• LLMNR and NBT-NS enabled
• Weak domain passwords or default passwords
• Lack of system patching (MS17-010, etc.)
• Unrestricted web administrator interfaces (Tomcat, Jenkins, etc.)
• Cleartext credentials cached in memory (dumping LSASS and analyzing, WDigest enabled)

Have you ever been duped by a phishing email?

Not that I am aware of!

What are some benefits of performing a manual SQL injection vs. an automated one like sqlmap?

For starters, sqlmap is a very noisy tool. If you are against any sort of network or web detection and rate-limiting defense mechanisms, manual injection attempts may be your only recourse. However, for tests against a test environment where the goal is to identify all vulnerabilities in a short time frame, tools such as sqlmap save a lot of time.

However, an automated tool works only as well as you tune it. If I suspect a parameter is vulnerable to SQL injection, I may run sqlmap against it for a cursory check with the debug flags on to show each GET/POST request and response from the web server. If sqlmap comes back that the parameter does not appear to be exploitable, manual analysis of each request and response is warranted to check whether sqlmap missed something, or the injection string requires additional finessing.

Do you prefer pen testing or red teaming?

I prefer red teaming. My personal passion is to develop payloads that bypass host-based protections and detection tools. This involves a lot of time spent analyzing the footprint created by the payload, which goes together with conducting a red team assessment.

What do you do when you hit a wall and can't seem to find any exploitable vulnerabilities?

When I hit a wall, I continue to dig for misconfigurations or oversights and get creative in my way to achieve a foothold. As an example, a co-worker of mine once hit a wall on a client's external penetration test. There were no easily exploitable vulnerabilities present on their external network. However, after a couple of days of intense reconnaissance, he discovered that one of the company's employees had posted developer meeting notes and videos on their personal GitHub, maybe by an accidental commit, or under the impression that it was a private repo. Combing through the videos, he discovered credentials on a whiteboard being used to discuss the application during its development phase. Sure enough, those credentials were used externally on the public application and allowed administrator access to the instance.

Sometimes, it is all about just getting a password. Once on an external network penetration test, I asked the client if I could send a single email to an employee, which was blatant phishing. The link wouldn't even work. The client was not sure why, but agreed. Inside that email, I embedded an HTML image using a UNC path to my rogue SMB server on the internet. These days, Outlook does not automatically render images from untrusted addresses unless an employee clicks to enable HTML content. However, if they forward the email, Outlook downloads the content, then forwards it. Oops. In this case, the target company was not blocking SMB (port 445) egress to the internet. The employee received the email, identified it as phishing, and forwarded it to their internal security team. The act of forwarding it caused the Outlook client to try and download the image via SMB (due to the UNC path), causing the employee's workstation to authenticate to my rogue SMB server to retrieve the image, resulting in the capture of the employee’s credentials hashed in NetNTLMv2 format.

Additionally, because the email was forwarded by an employee, it immediately became trusted content. Without being prompted to download HTML content, the Outlook clients of everyone on information security team who opened the email attempted to authenticate to my SMB server, harvesting their password hashes as well. The password hashes were then cracked offline and used to access external resources which were domain authenticated, or where employees reused their passwords.

Do you think your job will become harder or easier in the future?

This is a great question. I’m honestly not sure. The security space is growing. There are more solutions available, greater awareness and priority is being placed on securing systems, and progress is being made. I want to believe that my job will become harder as we all advance together—that is an exciting prospect. However, with the speed at which the tech industry operates, striving to push out new products and be the first to market, mistakes will inevitably be made. So for now, I do not see the space becoming easier or harder. It will continue to be a battle of learning enough to bypass the latest system or security control, relying on the technique for a bit of time, then having to develop new techniques, tactics, and procedures.