Rapid7 Blog

Brent Cook  



Metasploit: The New Shiny

It's been a while since I've written a blog post about new stuff in Metasploit (and I'm not sure if the editors will let me top the innuendo of the last one). But I'm privileged to announce that I'm speaking about Metasploit twice next month:…

It's been a while since I've written a blog post about new stuff in Metasploit (and I'm not sure if the editors will let me top the innuendo of the last one). But I'm privileged to announce that I'm speaking about Metasploit twice next month: once at the FSec 17 Conference in Varaždīn, Croatia September 7-8, and a second time at UNITED 2017, Rapid7's annual security conference in Boston September 11-14. The talk should be a wild ride through some of the interesting new features that Metasploit has gained over the past year, as well as amazing stuff we have underway for the next major version of Metasploit. With a project so large and varied, it can be challenging keeping it fresh and relevant. Amazing new open-source security projects pop up almost as fast as CVE allocations. Metasploit is definitely seeing a generational shift, with new developers coming in and older ones moving to new projects. As a result, we have done a lot of work this year moving Metasploit Framework to the next level, while preserving the things people love about it the most. Our 2017 Roadmap was just the beginning—we have a lot of interesting work on the horizon that will change how you think about Metasploit. I'm also helping with the Metasploitable3 CTF at the UNITED conference and helping run some Metasploit training. So if you have any questions about Metasploit, past, present, or future, this is your chance to get expert advice, either from me or from the five other Metasploit developers who will also be attending. It should be fun and educational, if not a little exhausting! Hope to see you there! Haven't yet signed up to join us at UNITED this year? Register here, or read more about some of the talks and features of this year's summit.

Metasploit Framework Valentines Update

Valentines day is just around the corner! What could be a nicer gift for your sweetie than a bundle of new Metasploit Framework updates? The community has been as busy as ever delivering a sweet crop of sexy exploits, bug fixes, and interesting new features.…

Valentines day is just around the corner! What could be a nicer gift for your sweetie than a bundle of new Metasploit Framework updates? The community has been as busy as ever delivering a sweet crop of sexy exploits, bug fixes, and interesting new features. Everyone Deserves a Second Chance Meterpreter Scripts have been deprecated for years in favor of Post Exploitation modules, which are much more flexible and easy to debug. Unfortunately, the Internet still abounds with blogs and other advice still recommending their use, and it is clear the word still hasn't gotten out. In a previous Metasploit release, we attempted an experiment removing all of the scripts that already had Post Exploitation modules. Unfortunately, this caused even more confusion since it looked like Metasploit was broken. Now, Metasploit will kindly suggest that users explore the vast world of Post modules instead. For now, all of the built-in Meterpreter scripts you know and love are back for one last dance, but you should really look at dumping those guys. Remember, there are many more Post modules in the sea! Traverse your Way into my Life With this release, we have a number of directory traversal updates, both offensive and defensive. First off, we have added a module for exfiltrating arbitrary data from a Cisco Firepower management console. The default credentials are also documented, so if you run into one of these in the wild, there is a good chance you can make a special connection. And in the "it's not you, it's me" department, Justin Steven has been busy finding and fixing a number of directory traversal bugs in Metasploit's session handler, that can be exploited if you interact with a rogue Meterpreter session. Of course you should practice "safe sess(ions)", but if you can't, update your Metasploit Framework and get protected. You Stole my Creds, my Phone, my Car, and my Heart If you're looking for credentials to add to your little black book, Metasploit release also adds credential extraction modules for Advantech WebAccess, Metrocontrol Weblog, and Cisco Firepower Management Console. And once you have filled your cred list, you can now manipulate them in a more powerful way thanks to improvements in credential management. Android Meterpreter adds a number of new features sure to make keeping up with your bae even easier (that doesn't sound creepy at all does it!) Android Meterpreter now supports stageless HTTPS, which makes it easier to keep your payloads secure, fast, and reliable. If you have trouble with your Android sessions falling asleep after you connect, keep them going all night (and day) long with the new wakelock command. Metasploit makes its first foray into car hacking with a new hardware bridge session type, along with a number of new modules for administering and exploiting OBD-II / CANbus networks in modern vehicles. But, it's not limited to just these, you can add your own hardware devices by implementing the HWBridge specification. Don't let your car spoil your next date, hack back! There are many more improvements and modules to enjoy as well, and they are all available now. So why not update your console with someone special, and make everyday a very special Metasploit Valentines day. For full details, see the latest detailed Metasploit release notes: https://community.rapid7.com/docs/DOC-3575

12 Days of HaXmas: Meterpreter's new Shiny for 2016

Merry HaXmas to you! Each year we mark the 12 Days of HaXmas with 12 blog posts on hacking-related topics and roundups from the year. This year, we're highlighting some of the “gifts” we want to give back to the community. And while these gifts…

Merry HaXmas to you! Each year we mark the 12 Days of HaXmas with 12 blog posts on hacking-related topics and roundups from the year. This year, we're highlighting some of the “gifts” we want to give back to the community. And while these gifts may not come wrapped with a bow, we hope you enjoy them. Editor's Note: Yes, this is technically an extra post to celebrate the 12th day of HaXmas. We said we liked gifts! Happy new year! It is once again time to reflect on Metasploit's new payload gifts of 2016 and to make some new resolutions. We had a lot of activity with Metasploit's payload development team, thanks to OJ Reeves, Spencer McIntyre, Tim Wright, Adam Cammack, danilbaz, and all of the other contributors. Here are some of the improvements that made their way into Meterpreter this year. On the first day of Haxmas, OJ gave us an Obfuscated Protocol Beginning the new year with a bang (and an ABI break), we added simple obfuscation to the underlying protocol that Meterpreter uses when communicating with Metasploit framework. While it is just a simple XOR encoding scheme, it still stumped a number of detection tools, and still does today. In the game of detection cat-and-mouse, security vendors often like to pick on the open source project first, since there is practically no reverse engineering required. It is doubly surprising that this very simple technique continues to work today. Just be sure to hide that stager On the second day of Haxmas, Tim gave us two Android Services Exploiting mobile devices is exciting, but a mobile session does not have the same level of always-on connectivity as an always-on server session does. It is easy to lose a your session because a phone went to sleep, there was a loss of network connectivity, or the payload was swapped for some other process. While we can't do much about networking, we did take care of the process swapping by adding the ability for Android meterpreter to automatically launch as a background service. This means that not only does it start automatically, it does not show up as a running task, and is able to run in a much more resilient and stealthy way. On the third day of Haxmas, OJ gave us three Reverse Port Forwards While exploits have been able to pivot server connections into a remote network through a session, Metasploit did not have the ability for a user to run a local tool and perform the same function. Now you can! Whether it's python responder or just a web server, you can now setup a locally-visible service via a Meterpreter session that visible to your target users. This is a nice complement to standard port forwarding that has been available with Meterpreter sessions for some time. On the fourth day of Haxmas, Tim gave us four Festive Wallpapers Sometimes, when on an engagement, you just want to know 'who did I own?'.  Looking around, it is not always obvious, and popping up calc.exe isn't always visible from afar, especially with those new-fangled HiDPI displays. Now Metasploit lets you change the background image on OS X, Windows and Android desktops. You can now update everyone's desktop with a festive picture of your your choosing. On the fifth day of Haxmas, OJ gave us five Powershell Prompts Powershell has been Microsoft's gift both to Administrators and Penetration Test/Red Teams. While it adds a powerful amount of capabilities, it is difficult to run powershell as a standalone process using powershell.exe within a Meterpreter session for a number of reasons: it sets up its own console handling, and can even be disabled or removed from a system. This is where the Powershell Extension for Meterpreter comes in. It not only makes it possible to confortably run powershell commands from Meterpreter directly, you can also interface directly with Meterpreter straight from powershell. It uses the capaibilites built in to all modern Windows system libraries, so it even works if powershell.exe is missing from the system. Best of all, it never drops a file to disk. If you haven't checked it out already, make it your resolution to try out the Meterpreter powershell extension in 2017. On the sixth day of Haxmas, Tim gave us six SQLite Queries Mobile exploitation is fun for obtaining realtime data such as GPS coordinates, local WiFi access points, or even looking through the camera. But, getting data from applications can be trickier. Many Android applications use SQLite for data storage however, and armed with the combination of a local privilege escalation (of which there are now several for Android), you can now peruse local application data directly from within an Android session. On the seventh day of Haxmas, danilbaz gave us seven Process Images This one is for the security researchers and developers. Originally part of the Rekall forensic suite, winpmem allows you to automatically dump the memory image for a remote process directly back to your Metasploit console for local analysis. A bit more sophisticated than the memdump command that has shipped with Metasploit since the beginning of time, it works with many versions of Windows, does not require any files to be uploaded, and automatically takes care of any driver loading and setup. Hopefully we will also have OS X and Linux versions ready this coming year as well. On the eight day of Haxmas, Tim gave us eight Androids in Packages The Android Meterpreter payload continues to get more full-featured and easy to use. Stageless support now means that Android Meterpreter can now run as a fully self-contained APK, and without the need for staging, you can now save scarce bandwidth in mobile environments. APK injection means you can now add Meterpreter as a payload on existing Android applications, even resigning them with the signature of the original publisher. It even auto-obfuscates itself with Proguard build support. On the ninth day of Haxmas, zeroSteiner gave us nine Resilient Serpents Python Meterpreter saw a lot of love this year. In addition to a number of general bugfixes, it is now much more resilient on OS X and Windows platforms. On Windows, it can now automatically identify the Windows version, whether from Cygwin or as a native application. From OS X, reliability is greatly improved by avoiding using some of the more fragile OS X python extensions that can cause the Python interpreter to crash. On the tenth day of Haxmas, OJ gave us ten Universal Handlers Have you ever been confused about what sort of listener you should use on an engagement? Not sure if you'll be using 64-bit or 32-bit Linux when you target your hosts? Fret no more, the new universal HTTP payload, aka multi/meterpreter/reverse_http(s), now allows you to just set it and forget it. On the eleventh day of Haxmas, Adam and Brent gave us eleven Posix Payloads Two years ago, I started working at Rapid7 as a payloads specialist, and wrote this post (/2015/01/05/maxing-meterpr eters-mettle) outlining my goals for the year. Shortly after, I got distracted with a million other amazing Metasploit projects, but still kept the code on the back burner. This year, Adam, myself, and many others worked on the first release of Mettle, a new Posix Meterpreter with an emphasis on portability and performance. Got a SOHO router? Mettle fits. Got an IBM Mainframe? Mettle works there too! OSX, FreeBSD, OpenBSD? Well it works as well. Look forward to many more improvements in the Posix and embedded post-exploitation space, powered by the new Mettle payload. On the twelfth day of Haxmas, OJ gave us twelve Scraped Credentials Have you heard? Meterpreter now has the latest version of mimikatz integrated as part of the kiwi extension, which allows all sorts of credential-scraping goodness, supporting Windows XP through Server 2016. As a bonus, it still runs completely in memory for stealty operation. It is now easier than ever to keep Meterpreter up-to-date with upstream thanks to some nice new hooking capabilities in Mimikatz itself. Much thanks to gentilkiwi and OJ for the Christmas present. Hope your 2017 is bright and look forward to many more gifts this coming year from the Metasploit payloads team!

Pokemon Go, Security, and Obsolescence

Pokemon Go started it. The crusty old house cell phone, which we had years ago ported from a genuine AT&T land line to a T-Mobile account, suddenly caught the attention of my middle son. "Hey Dad, can I use that phone to…

Pokemon Go started it. The crusty old house cell phone, which we had years ago ported from a genuine AT&T land line to a T-Mobile account, suddenly caught the attention of my middle son. "Hey Dad, can I use that phone to catch Pokemon at the park?" "Sure! Have fun, and don't come back until sundown!" A few minutes later, he had hunted down his first Pikachu, which apparently required running around the block in Texas summer heat a few times. Sweat-soaked but proud, he happily presented his prize. I could get used to this! The kids were getting out of the house, exploring the neighborhood, having fun, and I was getting a little peace and quiet. Then one day, Pokemon Go stopped working, stating that it did not support 'rooted' phones. First some back story. Our 'house phone' role is generally filled by the most-working last-gen reject device that is too old to be useful as a daily driver, but too new to throw away. In this case, it was a Google Nexus 4. I have always preferred the Google phones over other third parties for a number of reasons: They're cheap if you get the last generation (and sometimes the current). They usually lead the pack when it comes to software updates and hackability. However, given the industry's appetite for quick turnarounds and obsolescence cycles, (and in spite of Google's generally good support) this phone is end-of-life, and has not received an official firmware update in over a year. In fact, this phone is the amalgamation of two Nexus 4's, combined into a frankenstein assemblage of the most-working screen, battery, and charging ports of the original pair. Since it has been a year and a half since Google released a firmware for this phone, I had it running the next-best thing: Cyanogenmod 13, which backported Android 6 to this hardware. Now, this junker phone is up-to-date as much as the Android Open Source Project (AOSP) allows. But, there was now a show-stopper: you now can't run Pokemon Go on rooted phones using Cyanogenmod. Technically, there is a new set of hacks, but this is a cat-and-mouse game, but there comes a time in your life when you just want things to work. And they were already hooked. Why did Niantic decide to impose this restriction after several months of unrestricted access? It comes down to cheaters. People were rooting their phones specifically to fake GPS coordinates to get rare Pokemon, grow eggs, etc. Since having root access is also required to install non-stock firmware, in this guilty-until-proven-innocent model, we basically get to choose between two possibilities: get up-to-date software but sacrifice the ability to run some applications, or run increasingly out-of-date 'official' software, for the sake of satisfying a DRM or anti-cheating scheme. In the end, I decided that the stock firmware still allowed upgrading a lot of the key components via the Google's Play Store, the real core around which an increasing amount of the software in the Android ecosystem relies. Sure, I'm not getting the latest advances in encrypted filesystems, kernel hardening, or process isolation in the latest versions of Android, but it's a tradeoff. Maybe the phone will have died completely by the time the next exploitable bug in libstagefright rears its head. But, maybe it already has. It took over a year for enough of the moving parts for a reliable exploit for CVE-2015-3864, one of the 'StageFright' series of vulnerabilities, to come together within Metasploit. The exploit needed new payloads, new techniques, and a number of independent research projects to become useful outside of the proof-of-concept realm. In the end, it works very well, even better than the Metaphor exploit from earlier this year, and can be easily targeted to any vulnerable Nexus phone. Ironically, the very openness of the Google Nexus ecosystem made porting the exploit to those firmware builds particularly easy. In contrast, Samsung firmware, which contains many proprietary additions to the base Android system, and is not open-source, is harder to target simply because it is harder to examine. In spite of this, it was still possible to target Samsung phones as well. Effectively, with enough effort, any firmware is exploitable. It is just a question of time. When you think of exploits in the StageFright family, think of the vector: someone sends a special text message and take over a phone without anyone even reading it. You get an email, and without opening it, code is already executed on your device. It's a simple concept, but the fix is not nearly as straightforward. Automatic parsing of metadata in media files is a commonly-researched and targeted vulnerability in many different products. Adobe flash has had nasty vulnerabilities in its MP3 metadata parsing code earlier this year. Apple iOS has been vulnerable a number of times to similar attacks. Just last month, similar vulnerabilities in Android's libutils library were found, which could be attacked in a similar way. The exploit that we included in Metasploit for CVE-2015-3864 only targets one vector (web browser) and one file type (MP4 video files). However, there are many other vectors and file types that could also be exploited in the same family, that were discovered around the same time period as CVE-2015-3864. Not only that, but more vectors and file types have been found since the original round of StageFright branded vulnerabilities were hot in the news, and quietly patched. Of course, none of these patches have made it into the official firmware for my Nexus 4. I even had to do a double-take in researching this article, since Wikipedia claimed Android 5.1.1 was last updated 2 months ago, while I knew the phone hadn't gotten an over-the-air update in some time. To really know if you're up-to-date, you have to look at the build number, Nexus 4 being on LMY48T while the latest is LMY49M. It's unlikely that the average consumer with a phone running Android '5.1.1' would be able to know difference between a vulnerable or up-to-date build number, much less the average business with a bring-your-own-device policy. The choice between running the software you want, like Pokemon Go, and the quick road to obsolete devices in the Android ecosystem, at best forces users to make a choice between security and functionality. The theoretical exploit chains being patched this year can easily turn into next year's reliable Metasploit module. Maybe it's time to bring back to a land line.

Important security fixes in Metasploit 4.12.0-2016091401

A number of important security issues were resolved in Metasploit (Pro, Express, and Community editions) this week. Please update as soon as possible. Issue 1: Localhost restriction bypass    (affects versions 4.12.0-2016061501 through 4.12.0-2016083001) On initial install, the Metasploit web interface displays…

A number of important security issues were resolved in Metasploit (Pro, Express, and Community editions) this week. Please update as soon as possible. Issue 1: Localhost restriction bypass    (affects versions 4.12.0-2016061501 through 4.12.0-2016083001) On initial install, the Metasploit web interface displays a page for setting up an initial administrative user. After this initial user is configured, you can login and use the Metasploit web UI for the first time. Since this initial screen is unauthenticated, it can only be accessed via a local user (e.g. hitting the localhost hostname or loopback IP address Until the most current release, the initial setup page access restriction does not work properly in Metasploit 4.12.0 releases. Instead, on initial install, the page for setting up the initial administrative user is accessible from all addresses on the host running Metasploit.  An attacker might be able to 'race' a fresh Metasploit installation and become the first to create an administrative user. Mitigation: For users who are planning on using Metasploit with the web interface, it is important to isolate the machine from hostile networks until initial configuration is complete, or be sure to use the latest Metasploit installer in which this issue is resolved. Thanks to Brandon Perry for discovering and reporting this issue. Issue 2: Predictable session cookies    (affects versions 4.12.0-2016061501 through 4.12.0-2016083001) Metasploit uses a randomized secret key to protect session cookies from forgeries. On installation, it randomizes the secret key and stores it in a local configuration file. As of Metasploit 4.12.0, the update packages inadvertently include a static version of this secret key file, which overwrites the randomly-generated one. The effect of this is that some Metasploit installations will all have the same hard-coded base session token, leading to forgeable session cookies, allowing an unauthenticated user to perform remote code execution via another object deserialization bug. Mitigation: On startup, Metasploit will identify 'bad' static secret keys that may be installed, and if found, the base secret key is regenerated. If this fix is needed, and if a user is applying the latest update via the web UI, the UI may appear to hang during the update, though it will complete successfully in the background. In this case, simply refresh the web UI after 10-20 minutes. If it loads a login screen, the update applied successfully. Users who updated from 4.11.0 or earlier builds are not affected, but are still encouraged to update. Thanks to Justin Steven for discovering and reporting this issue. Issue 3: config.action_dispatch.cookies_serializer is set to :hybrid    (affects versions 4.12.0-2016061501 through 4.12.0-2016083001 Metasploit versions 4.11.x and earlier use the default 'marshal' cookie type, which is vulnerable to remote object instantiation / remote code injection for a user who has the ability to generate a signed session cookie. Mitigation: The Metasploit 4.12.0 point release switched to the 'hybrid' type, which gives an update path for users to the safer 'json' type. The latest release switches entirely to 'json' cookie serialization method. Thanks to Justin Steven for discovering and reporting this issue.

Metasploit Weekly Wrapup for 2015-10-15

Welcome to this week's Metasploit Wrapup. I'm your host Brent Cook, tagging in for egypt who just finished speaking about Metasploit at the Texas DIR Telecommunications Forum. This week was largely focused on bug fixes and refinements. In the fixes bucket, PowerShell sessions now properly…

Welcome to this week's Metasploit Wrapup. I'm your host Brent Cook, tagging in for egypt who just finished speaking about Metasploit at the Texas DIR Telecommunications Forum. This week was largely focused on bug fixes and refinements. In the fixes bucket, PowerShell sessions now properly upgrade with the 'sessions -u' command. Fixing this also revealed some general problems handling PowerShell commands, which were also fixed. SRVHOST, like LHOST now supports tab completion, which is super useful rather than having to remember what your local IP addresses are.  Modules using SSL can now set advanced options, including support for TLS 1.2, and a similar fix was applied to SMB and TCP login scanner modules. We also fixed a bug preventing 64-bit Linux staged command payloads from running, which unlocks loading some more interesting 64-bit Linux payloads in the future. As modules get used in more varied scenarios and environments, deficiencies in modules are often uncovered. As a result of your reports, MS SQL, IMAP and POP3 protocol handlers now handle network failures better. The Java RMI scanner is also more resilient when handling larger protocol responses. Even the venerable msfd now has a 'quiet' option, that makes it work nicely with dumb network clients. Of course, there were a few new modules this week as well, including: ManageEngine ServiceDesk Plus Arbitrary File Upload by Pedro Ribeiro Watermark Master Buffer Overflow (SEH) by Andrew Smith and metacom HP SiteScope DNS Tool Command Injection by Kirk Hayes, Charles Riggs and Juan Vazquez

Metasploit Framework Tools Reorg

There are a wide variety of interesting and useful tools in the Metasploit Framework. Many of these are available from the top-level of Metasploit in the form of modules and library code. You can find countless tutorials and blogs about how to put msfconsole, msfvenom…

There are a wide variety of interesting and useful tools in the Metasploit Framework. Many of these are available from the top-level of Metasploit in the form of modules and library code. You can find countless tutorials and blogs about how to put msfconsole, msfvenom and other top-level commands to good use. However, not many people know about the 'tools' directory, which contains many useful, single-purpose scripts, with topics spanning from exploit development to statistics. One of the problems with the tools directory is that it was not very well organized. Like a messy toolbox, it had grown organically over the years, making it difficult to find things. To correct this, we have reorganized the tools directory by category, making tools easier to discover and encouraging their use. The new categories are: dev: tools for managing developer tasks exploit: tools for developing exploits modules: tools for gathering project statistics, checking code quality, and checking payloads password: tools for extracting and cracking passwords recon: tools for collecting target data In the process, we found a few tools that may have outlived their usefulness. While we have not deleted anything yet, we may remove some obscure or unused tools in the future. Until then, please take a moment to check out the new cleaned-up tools directory and find something new! While you're there, be sure to check out our latest tool addition, MSU Finder.

Metasploit Framework Open Source Installers

Rapid7 has long supplied universal Metasploit installers for Linux and Windows. These installers contain both the open source Metasploit Framework as well as commercial extensions, which include a graphical user interface, metamodules, wizards, social engineering tools and integration with other Rapid7 tools. While these features…

Rapid7 has long supplied universal Metasploit installers for Linux and Windows. These installers contain both the open source Metasploit Framework as well as commercial extensions, which include a graphical user interface, metamodules, wizards, social engineering tools and integration with other Rapid7 tools. While these features are very useful, we recognized that they are not for everyone. According to our recent survey of Metasploit Community users, most only used it for the open source components, preferring to use the command-line tools over the graphical ones. Also, while we do our best to ensure that Metasploit Community and Pro releases are of high quality, they are not always supplied with the latest hot new exploits and payloads available in Metasploit Framework. While it has always been possible to simply setup a development environment and run the latest metasploit-framework code from github directly, it can still be tricky to setup and keep up to date. Kali Linux 2.0 now publishes the open source pieces of Metasploit Framework with its distribution, but the release schedule still follows that of Metasploit Community / Pro editions, and it of course does not necessarily help those who prefer other operating systems. To address the needs of open source enthusiasts, those needing more frequent updates, or those simply looking for an easy way to setup a database for Metasploit Framework development use, we have created Open Source installers for Metasploit Framework for Windows, OS X and Linux x86 and x86-64 platforms. These installers utilize the Omnibus tool from chef in order to package everything needed to run Metasploit Framework, from dependent libraries, specific Ruby versions up to a built-in PostgreSQL database. The installers are easy to install and get up and running in seconds. They are also built and tested automatically each night, so you can always run 'msfupdate' and get the latest exploits and payloads without having to setup a development environment. The installers also integrate with your OSes native package manager, be it Linux RPM or DEB-based, MSI for Windows or PKG for OS X. That makes them easy to uninstall as well. For information about how to install and use these new packages, see our wiki page on the Metasploit Framework project github project. The installers themselves are also open source. So if you see a problem, pull requests or issue reports are very welcome! Note that in addition to these Metasploit-specific installers, there are other ways to get Metasploit Framework, such as through Dave Kennedy's PenTester Framework or even pre-installed in Kali Linux. The Metasploit Framework omnibus installers provide another way to get the open source Metasploit Framework running on a variety of platforms quickly and easily.

Unicode Support in Meterpreter

A short, mostly-accurate history of character encodings In the beginning, when you wanted to use a computer to store text, there were not many options - you inherited something from punchcards like EBCDIC or invented something convenient and unique to your system. Computers did not…

A short, mostly-accurate history of character encodings In the beginning, when you wanted to use a computer to store text, there were not many options - you inherited something from punchcards like EBCDIC or invented something convenient and unique to your system. Computers did not need to talk to each other, so there was not much point in standardizing between vendors. Things were pretty simple. Then, there came the need for computers and vendors to interoperate and communicate. Thus, ASCII and later some of its cousins like ISCII, VISCII, and YUSCII were born. And now, large islands of computers, teletypes, modems, etc. could communicate with each other. ASCII only defined the lower 7 bits of a character, so other non-english encodings frequently used the upper half to define the extra language-specific characters. Things were a little confusing on the edges, but largely compatible, as long as you stuck to the lower 7-bits. To deal with alternate languages MS-DOS, OS/2 and 16-bit Windows used a concept called code pages to define how the display characters from different languages. This was somewhat consistent as long as you stuck to Microsoft products (they were not a real standard), and as long as you did not need to look at more than one language at a time. Of course, there was still a limit on the number of characters one can represent with the extra half-byte in ASCII. Various multi-byte 'encodings' like Big5 and Shift JIS were invented to represent eastern languages like Chinese and Japanese, which can have thousands of characters. These were all largely incompatible and still did not work if one wanted to use more than one language. The term 'mojibake' describes some of the hilarious results that can occur when incompatible encodings are interpreted the wrong way. Things started getting really confusing. Eventually, a new standard called Unicode was born to deal with the mess of these incompatible language encodings.  It defined the concept of code points, which are numerical representations of all the characters, ligatures, symbols, and other doodads from the world's languages, along with ways to store and transmit them between computers. Fast forward to the present: if you're an Android, iOS, Mac, Linux, BSD, or Unix user, you're probably used to Unicode working out of the box. This is thanks to a simple exchange and encoding standard called UTF-8. Being fully compatible with ASCII text, it does not necessarily require a lot of special support from existing programs in order to work. This is especially true in languages like C, where a UTF-8 string looks just like a C string. Sure, concepts like 'uppercase' and 'isalpha' take on new meaning and require thousands of lines of code to get right, but it is generally possible to write a program that ignores the concept of Unicode but still works with it without issues with UTF-8 strings. Now, let's go back to around the time that the Unicode standard was born. Prior to UTF-8 existing, there was an encoding standard called UCS-2 that used 16-bit words (the WCHAR or wide character of you have done any Windows programming) to represent international characters. This had benefits for efficiency: doing things like finding the number of characters in a UCS-2 string is just as easy as a traditional C string. The designers of Microsoft Windows NT decided to adopt this fledgling standard and designed the whole system to be compatible with UCS-2. Of course, to make it compatible with all the C code in the world that did not understand wide characters and still wanted to use code pages and weird encodings, Windows also was designed with a compatibility layer that supports those too. Why 2 bytes? Well, 4 was seen as too much wasted space, and after all, 65535 characters should be enough for anyone. As you can probably guess, it did not take long to determine that 16 bits was not actually sufficient to actually encode the world's character sets. Even efforts to merge common Japanese and Chinese characters, called Han Unification, did not yield enough space to represent all of the known characters (to say nothing of Emoji !) So, the Unicode standard grew, Windows switched to a new UTF-16 encoding which allowed using multiple 16-bit words to represent the extended characters. Unfortunately, UTF-16 that has none of the speed benefits of UCS-2 and none of the compatibility or size benefits of UTF-8. Effectively, internationalization in Windows is based on a mixture of different encoding standards, old, new and transitional, where programs can opt in or out of seeing all characters by explicitly enabling Unicode, or only some by living in an ASCII or code-paged world. And today Where does this tie in to the Metasploit Framework? Meterpreter now has Unicode Support on Windows. Why so long in coming? Effectively, Meterpreter was originally built as an ASCII-native C program that uses byte-size characters, taking advantage of the legacy support for ASCII and code pages in Windows. Flipping the switch wholesale to UTF-16 would break a number of things. The Ruby language (which the Metasploit Framework is based on) has gone through some growing pains as well with Unicode, but it is now a stable and first-class citizen. Thankfully, since UTF-8 support in Ruby 2 is actually very good (some might say over-zealous!), and since Meterpreter was already dealing in C strings, switching to UTF-8 was largely painless. In fact, once the Unicode filters were turned off in the Metasploit Framework, the Posix, PHP and Python meterpreters worked largely unmodified. On Windows, some string gymnastics are applied to convert from UTF-8 on the wire to UTF-16 which Windows is now based on. The Java language's support for Unicode largely mirrors Windows (it uses UTF-16 internally), but there is a Pull Request to add support there as well. Here are some examples of its usage - it is largely transparent: meterpreter > ls Listing: e:\metasploit-framework\mytest\test ============================================ Mode Size Type Last modified Name ---- ---- ---- ------------- ---- 40777/rwxrwxrwx 102 dir 2015-03-20 15:43:52 -0500 . 40777/rwxrwxrwx 1564 dir 2015-03-23 09:50:17 -0500 .. 100666/rw-rw-rw- 15 fil 2015-03-17 02:50:27 -0500 プロトタイプ.txt2 meterpreter > mv プロトタイプ.txt2 ʇıoʃdsɐʇǝW.txt meterpreter > ls Listing: e:\metasploit-framework\mytest\test ============================================ Mode Size Type Last modified Name ---- ---- ---- ------------- ---- 40777/rwxrwxrwx 102 dir 2015-03-24 09:42:53 -0500 . 40777/rwxrwxrwx 1564 dir 2015-03-23 09:50:17 -0500 .. 100666/rw-rw-rw- 15 fil 2015-03-17 02:50:27 -0500 ʇıoʃdsɐʇǝW.txt As a side note, the ls command gained a few new tricks of its own, such as viewing Windows 'Short' names for compatibility with older operating systems and MS-DOS: meterpreter > ls -x c:\ Listing: c:\ ============ Mode Size Type Last modified Short Name Name ---- ---- ---- ------------- ---------- ---- 40777/rwxrwxrwx 0 dir 2015-02-19 03:31:32 -0600 $SYSRE~1 $SysReset 40777/rwxrwxrwx 0 dir 2013-08-22 03:45:52 -0500 DOCUME~1 Documents and Settings 40555/r-xr-xr-x 0 dir 2015-03-17 04:29:10 -0500 PROGRA~1 Program Files 40555/r-xr-xr-x 0 dir 2015-03-19 10:48:07 -0500 PROGRA~2 Program Files (x86) 40777/rwxrwxrwx 0 dir 2015-03-18 08:21:01 -0500 PROGRA~3 ProgramData 40777/rwxrwxrwx 0 dir 2015-03-24 06:57:11 -0500 SYSTEM~1 System Volume Information 40777/rwxrwxrwx 0 dir 2015-03-17 04:33:11 -0500 METASP~1 metasploit As well as new sorting options. meterpreter > ls -h Usage: ls [dir] [-x] [-S] [-t] [-r] -x Show short file names -S Sort by size -t Sort by time modified -r Reverse sort order There are still other areas of the Meterpreter we are looking at to improve Unicode support, such as registry access and user enumeration, but this is a good first step. Have fun using Unicode in meterpreter. A big thanks to OJ, HD and @zeroSteiner and @schierlm for contributing fixes, testing time and reviews.

12 Days of HaXmas: Maxing Meterpreter's Mettle

This post is the twelfth in a series, 12 Days of HaXmas, where we usually take a look at some of more notable advancements and events in the Metasploit Framework over the course of 2014. As this is the last in the series, let's peek…

This post is the twelfth in a series, 12 Days of HaXmas, where we usually take a look at some of more notable advancements and events in the Metasploit Framework over the course of 2014. As this is the last in the series, let's peek forward, to the unknowable future. Happy new year, it's time to make some resolutions. There is nothing like a fresh new year get ones optimism at its highest. Meterpreter is a pretty nifty piece of engineering, and full of useful functionality. The various extensions and delivery mechanisms can do amazing things that I am still trying to wrap my head around. But, there are other, more fundamental, parts of Meterpreter's design that are showing their age. The primary problems I have run into are that it's not terribly efficient and still a little unpredictable when stressed, the POSIX meterpreter is pretty tricky to build and leads to a lot of frustration, and testing is a somewhat manual, tribal-knowledge sort of process. Due to the high degree of integration between Meterpreter, its wire protocol (TLV) and Metasploit Framework, it can be difficult to make sure that all of the bells and whistles keep working, either in isolation or together, and difficult to track down the cause of a problem. Luckily, my job is to make all of these things better this coming year! To this end, I have been spending my first month at Rapid7 experiencing all of the pain and joy of Meterpreter development.  At the same time, I have been hatching plans to make Meterpreter an even more amazing tool. As part of this effort, I started working on an experimental tool that I'm calling 'mettle'. Mettle means to deal with difficulties in a spirited and resilient way. There also happens to be a cool looking Marvel comic book character.  It is currently just something I am working on to scratch my second-system itch, but it tries to solve a few real problems with Meterpreter's design today. If they can build it, they will come First up, the POSIX build system for meterpreter has not seen as much love as the Windows version. Its understandable: working on Makefiles is not very sexy work, and there is nothing like a lot of technical debt in a build system to sap one's spirits.  But, they are important! So, the first leg of my project is to put together a build system that can work with multiple architectures, with cross compilation and readability in mind. Rule number one is to require nothing fancy, just GNU make. Everything else can be built or bootstrapped. This is my top-level make file so far: PROJECT_NAME:=mettle ARCH=i386 TARGET:=i386-linux-eng include scripts/make/Makefile.common include scripts/make/Makefile.libev include scripts/make/Makefile.libpcap include scripts/make/Makefile.libressl include scripts/make/Makefile.libtlv include scripts/make/Makefile.kernel-headers include scripts/make/Makefile.tools There is not a lot to see here - almost everything is in the include files. The point of this is to make the build system easier to understand by not including the kitchen sink in a single big file. This also makes merging changes from pull requests easier, since unrelated changes don't bleed into each other. Want to add a new library? It's one line here, and all of the goo goes somewhere else.  Experimenting with several of the older PRs for meterpreter, all of them conflicted heavily with the big main Makefile, leading to a merge nightmare. Separated-out code is easier to merge and test, and helps prevent bit-rot. The other thing to note is that there is an addition of a 'tools' and a 'kernel-headers' makefile.  I'm working on building everything with a cross-compiler, notably the ELLCC compiler with the musl C library. ELLCC is an interesting project because it provides a single toolchain that not only targets many different platforms, but itself runs on those same platforms and more. Musl is a C library focused on security and simplicity. It also happens to still support Linux 2.4 kernels, which makes it interesting for 'run everywhere' software like Meterpreter.  By using a cross compiler, builds on my Ubuntu 14.04 system can generate the same binaries as your Fedora 20 system. But, you could also build them on OS X or Windows and get the same results. It also makes it possible to target other architectures and platforms later. I have an outer driver script that looks like this, just to keep things honest: #!/bin/sh for i in armeb-linux-engeabi armeb-linux-engeabihf \ arm-linux-engeabi arm-linux-engeabihf \ mipsel-linux-eng mipsel-linux-engsf \ mips-linux-eng mips-linux-engsf \ ppc-linux-eng \ i386-linux-eng x86_64-linux-eng; do echo Building target $i make TARGET=$i; done Each of the builds is isolated into its own build tree, so you can keep them around and switch between them without having to 'make clean' first: ~/projects/mettle$ ls builds/ armeb-linux-engeabi armeb-linux-engeabihf i386-linux-eng kernel-headers-3.12.6 ~/projects/mettle$ ls builds/i386-linux-eng/ include lib libev libpcap libressl share I'm also keeping the source tarballs for the build in a separate repository to provide both a fast and reliable mirror for upstream sources. This removes dependency on upstream sources that may move, become unavailable, or might even be replaced. ~/projects/mettle/deps$ ls ellcc-Mac_OS_X_10.9.5-0.1.6.tar.gz kernel-headers-3.12.6.tar.xz libpcap-1.6.2.tar.gz ellcc-x86_64-linux-eng-0.1.6.tgz libev-4.19.tar.gz libressl-2.1.3.tar.gz Separating out the pieces I'm a big fan of separation of concerns and event-oriented programming. Modular code is simpler and easier to test because you can test each part in isolation. It also makes it easier to change one piece without affecting everything, because the interfaces are isolated and mockable. On that note, the core Meterpreter command dispatcher looks something like this (simplified, but not a lot): The server_dispatch function runs in a loop, receiving packets from the control socket. A 'packet' in this case is really just a header followed by a command. Each packet is passed to command_handle, which then scans a command table of strings to find a pointers to a function that knows how to parse and operate on this kind of packet. In most cases, a new thread is created by command_handle, which is passed the function pointer and the packet. The packets are then passed to the handler function, which may be built into Meterpreter or dynamically loaded as part of an extension (usually the later). While this design the listening socket from blocking while a command is dispatched, it means that a post module like windows/gather/enum_services translates into hundreds of threads spawned within Meterpreter. The lack of synchronization between these threads can also translate into stability problems, as a command sequence might be sent as 'open handle, close handle', but actually get executed in reverse. This pass-through design also means that every layer of the stack has to understand the concept of a 'packet', how to parse it and how to generate responses directly back to the command socket. This makes it difficult to re-purpose the command handlers for a different API, like being hooked to a scripting language, or using a different type of wire format to communicate from the Metasploit framework. To this solve these problems, I am integrating an event loop into mettle's design that will separate the command packet parsing from the callbacks, and separates the execution of the command handler itself into a separate thread. This design has several advantages. First, there only needs to be two threads at minimum - one to handle command ingress and egress, and another to actually execute the commands. More command dispatchers could be added for multiple channels into mettle. Second, the packet parsing and generation will be separated out from the actual API calls. This will allow using the APIs from other sources, such as a built-in scripting language perhaps, or a different command format than TLV if desired. Third, it allows commands to be run sequentially and synchronously without blocking the communications channels. Commands could even continue executing in the event of a disconnect from the command channel back to the Metasploit framework, and this design would be able to continue without any issues. It would behave similarly to how command dispatch works within the framework itself, supporting the concepts of queued commands and retries as well. Designing for testability I'm tackling this on two fronts. On one hand, Meterpreter has some lovely test cases already in the form of the post/test/ modules. Unfortunately, these probably are not run as frequently as they should be, and are not part of our continuous integration tests. As such, they have had a little bit rot. But, I have been working on cleaning these tests up and fixing issues in Meterpreter that prevent these tests from passing today. I have also been working in Jenkins and vagrant to provision fresh victim VMs on the fly, run rc scripts with the Metasploit. Tests are nice, automated tests are awesome! Hopefully, it will not take too long to get all of these tests passing reliably with Meterpreter, and I these tests have already found some interesting bugs, such as this. Because Meterpreter is already largely built as shared library objects, this is actually an ideal setup for unit testing. One 'simply' links a test harness executable to the libraries and away you go. However, there is a problem as hinted at above: the actual API calls parse and return Packets directly, rather than implementing a C calling convention. The more modular design of mettle should help this by giving the standard API calls a real function signature that can be tested directly, so the transport parsing, dispatch and APIs themselves can be tested separately. This will also allow mocking up the API backend itself, to make concepts like record and playback meterpreter sessions for testing Metasploit post modules possible without a backing VM. If not anything else, 2015 should be the year of tests! Onward, onward There is a lot more work to do on mettle, so things may change over time. I envision it possibly replacing or living along-side the POSIX Meterpreter first, since the API breadth is lower there compared to the Windows version. It might simply live along-side Meterpreter in general, since it would be nice to have something that could be used for research and development without as much worry about breaking things in the short term. I'm hope your mind is also racing like mine, thinking of other uses for an event loop and disconnected command queue, like 'sleep' modes, scriptability, covert channels and similar. But for now, I am continuing with the basics of getting it as testable, buildable, efficient and reliable as possible.

12 Days of HaXmas: Does it Blend Like a Duck?

This post is the fifth in a series, 12 Days of HaXmas, where we take a look at some of more notable advancements and events in the Metasploit Framework over the course of 2014. Writing portable software is not hard. It's just like walking through…

This post is the fifth in a series, 12 Days of HaXmas, where we take a look at some of more notable advancements and events in the Metasploit Framework over the course of 2014. Writing portable software is not hard. It's just like walking through a minefield! Getting to the other side, that's the tricky part. Sure, if you target C, Unix-like systems and GCC or LLVM, you may not run into too many hassles these days. There are still a few annoying differences between BSDs and Linux, but POSIX and better compiler compatibility have been shrinking the gaps. Even the old Unix systems have learned to either adapt or die. Endian and byte-alignment considerations are becoming easier too, with newer ARM, MIPS and PowerPC CPUS are starting to copy the behavior of that the lovable little-endian, byte-access-agnostic x86. Though don't forget to test occasionally on a SPARC machine, just to keep you honest . But even if you restrict your program to use only POSIX systems and library calls, and your code builds without warning and -Wpedantic on a half dozen compilers, beware the silent killer - error handling. Two system calls may act the same way when everything is ok, but do they both fail the same way?  You have to not only make sure that your API calls quack like a duck and walk like a duck, but also blend like a duck. Quiet compilers have a way of creating a false sense of security. There are plenty of traps. Take asprintf() for instance. It attempts to make formatting string buffers safe and easy, but has 2 ways it can fail, either by returning an integer or setting a pointer. Either (or both) of which are these may be set by different implementations. Or, consider poll, which has a history of not handling errors correctly (Window's WSAPoll failure) or even the checks for broken poll implementations being broken (OS X's poll() on python). I recently merged Windows support for LibreSSL, and making things work properly was just as much challenge as making them fail properly. There was an additional challenge as well! The LibreSSL team wanted to add no new #ifdefs to the upstream codebase, lest they make the code harder to audit and test. My eventual goal is to incorporate this work into Meterpreter, updating the aging, yet nicely Heartbleed-free, OpenSSL 0.9.8 that it currently uses. Easy things first: Random Number Generation Windows was ahead of its time in this area. When the OpenBSD team added the getentropy() syscall earlier this year in order to provide a failure-free method of seeding arc4random() (the random number generator for LibreSSL), it turned out that acquiring entropy on many popular OSes was much harder than it needed to be. With Windows, it was extremely simple thanks to CryptGenRandom: if (CryptAcquireContext(&amp;provider, NULL, NULL, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT) == 0) goto fail; if (CryptGenRandom(provider, len, buf) == 0) { CryptReleaseContext(provider, 0); goto fail; } CryptReleaseContext(provider, 0); Some day, when arc4random becomes a POSIX standard and Windows is maybe the odd-OS-out, I'll have a different perspective, but for now, it was a pleasant surprise. But, shortly after, we run into the first porting challenge: Problem 1: Sockets vs File Descriptors With POSIX, file descriptors are a common interface for many things, and thus things like sockets, terminals, disks, and files can be manipulated with the same basic system calls like open(), read(), write(), and close(). Windows was designed in a parallel universe and chose the HANDLE as the common abstraction, and if you like handles can use something like OpenFile(), CloseHandle(), etc. There are some alter-ego functions like _open(), _read(), _write() exposed as well that operate on POSIX-like file descriptors, but their scope is far from universal. Socket APIs in Windows add to the mix if incompatible APIs. While a POSIX programmer might not even blink to write a function that generically takes an 'int' and works on both sockets and files, Windows Sockets look like an 'int' to the compiler, but have their own subset of APIs that only work on sockets, e.g. recv/send/closesocket. Vice-versa, APIs that work on files in POSIX environments don't work on Windows sockets. Take the following contrived example: void echo4(int accept_fd) { char buf[4]; int sock = accept(accept_fd, NULL, NULL); if (sock >= 0 && read(sock, buf, sizeof(buf)) == 4) write(sock, buf, sizeof(buf)); else perror("accept or read"); close(sock); } This code compiles cleanly on Linux and Mingw-w64 , but it behaves completely differently. In a POSIX system, it will accept a connection on listening socket, read 4 bytes, echo them back and close the connection. On Windows, read() and write() fail and there is a resource leak. The main problem is that accept() on Windows does not return an int, it returns a SOCKET, which the compiler will happily cast to an int anyway, since that's what it is under the covers. But, since read, write and close do not work on SOCKETs, you never know about it (except for the program not working). Or worse, you used recv and send originally, but the close() call is now silently leaks SOCKET objects. The usual answers to this problem are #ifdefs or using Cygwin, but those are either ugly, or prevent it using LibreSSL as a native Windows port. My solution was to use the C preprocessor to create alternate versions of IO functions that automatically do the right thing for sockets and file descriptors. static inline ssize_t posix_read(int fd, void *buf, size_t count) { ssize_t rc = recv(fd, buf, count, 0); if (rc == SOCKET_ERROR) { int err = WSAGetLastError(); return err == WSAENOTSOCK ? _read(fd, buf, count) : wsa_errno(err); } return rc; } #define read(fd, buf, count) posix_read(fd, buf, count) This defines a shim posix_read function that exploits a the failure behavior of socket functions. We can then play a guessing game! If the argument was actually a SOCKET (remember, the compiler won't tell you if you used the wrong function), we have guessed the correct call. If it isn't, the error code WSAENOTSOCK tells us to use the read function defined for file descriptors instead. Problem 2: Error Handling Have you ever had a program popup a dialog saying "Error: Success" ? POSIX functions set errno on failure, but Windows socket functions instead set an internal variable accessible via WSAGetLastError(). But, errno is still available, so in our example code, using our shim function for 'read' may still report 'Success' for errno, since that is what the 'perror' function looks for. To fix errno's behavior, we define a mapping function that converts WSA* errors into errno-style constants. No more 'ESUCCESS'! : static int wsa_errno(int err) { switch (err) { case WSAENOBUFS: errno = ENOMEM; break; ... But we're not out of the woods yet. Windows 'strerror' function does not understand how to format all of the error codes defined in errno.h. So, I overrode that as well: static inline char * posix_strerror(int errnum) { if (errnum == ECONNREFUSED) return "Connection refused"; return strerror(errnum); } #define strerror(errnum) posix_strerror(errnum) An alternative could have been to remap back to the original WSA error code, and to use FormatMessage to get the localized message. But, these messages turned out to be far longer than the fixed-length error buffers that OpenSSL/LibreSSL allocate for error messages. Rather than have messages like 'The connection that you were about to use has...', I opted for the I18N-insensitive hack. Problem 3: select() The Windows and POSIX APIs for signaling on events are both very capable, but unfortunately much different from each other. The Windows socket API throws portable software a bone by implementing select(), but this is far from POSIX compatible. The catacombs of stackexchange are littered with the bones of programmers surprised that Windows select doesn't work on files, pipes, the console or anything that is not a socket. Surprisingly, even high-level languages like Perl and Python do not bother to hide this difference. Even if one sticks to sockets, select on Windows has some surprising properties to a POSIX programmer. It is possibly even a little more secure! Among the many design flaws of the select interface in general, it takes a fd_set structure that indicate the sockets one is interested in waiting on. On POSIX systems, this is a actually a bitmap, one bit per descriptor. So, if your descriptor value is 1000, bit 1000 has to be set in the fd_set. This can lead to failures and buffer overflows if you have a file descriptor value larger than the fd_set structure itself, even if only a single descriptor is actually set. On Windows, an fd_set is instead an array of SOCKETs. That means that the literal value of the SOCKET, when converted to an integer does not matter, though it is limited to about 32 entries before the fd_set is full. For small numbers of sockets, the Windows interface actually helps prevent the overflow! Of course, select is still bad for performance reasons, so we want to fix it. For readability and security, first we converted all select(2) calls in LibreSSL into poll(2) calls. Poll is a little better than select on POSIX systems, and has the advantage of not having the fd_set overflow issue as well. Windows has a poll-like function as well, but it has the same limitations as select - it only works on sockets. To fix this, I implemented a poll implementation that uses the same mother-may-I SOCKET testing technique to determine if we should do select or WaitForMultipleObjects on passed in values. Here is the code that helps determine what's what: static int is_socket(int fd) { WSANETWORKEVENTS events; return (WSAEnumNetworkEvents((SOCKET)fd, NULL, &events) == 0); } There are some extra heuristics to determine if a pipe or a console handle have been passed in. In the case of a Windows console, we do not want poll to tell the caller that stdin has data available, when instead it just signaled on a mouse or window event. Because stdin is always blocking, performing a read() without data will hang a program. Since we don't care about these extra events from a console program, poll() automatically discards the unneeded events from the queue: /* * Check if this file is stdin, and if so, if it is a console. */ if (h == GetStdHandle(STD_INPUT_HANDLE) && PeekConsoleInput(h, &record, 1, &num_read) == 1) { /* * Handle the input console buffer differently, * since it can signal on other events like * window and mouse, but read can still block. */ if (record.EventType == KEY_EVENT && record.Event.KeyEvent.bKeyDown) { rc |= POLLIN; } else { /* * Flush non-character events from the * console buffer. */ ReadConsoleInput(h, &record, 1, &num_read); } } else { rc |= POLLIN; } You can view the whole lovely thing here complete with comments and rationales. I'm surprised that a function like this is not more commonly available in other programs. Problem 4: Header files It should come as no surprise that the standard headers for Windows are also different from POSIX systems. On Windows, the standard headers seem to be optimized for precompiled header usage, and thus include the kitchen sink. In my original example, the header section looked like this: #ifdef _WIN32 #include <ws2tcpip.h> #else #include <sys/socket.h> #include <sys/types.h> #include <sys/uio.h> #include <unistd.h> #endif With the #ifdef abolition, and because I'd rather spend 10 minutes doing a 1 minute job automatically, I added aliases for all of the missing POSIX headers that do the correct thing. The magic headers for sys/ioctl.h looks like this: #ifndef _WIN32 #include_next <sys/ioctl.h> #else #include <win32netcompat.h> #define ioctl(fd, type, arg) ioctlsocket(fd, type, arg) #endif The build system then puts these include files higher in priority in the compiler's include path. The magic #include_next directive tells the compiler to look for the next file in the search path, making this a no-op on a system with a system-wide <sys/ioctl.h>. After a few dozen of these, the porting job was complete. This saved rework on dozens more .c files that otherwise would have had ugly Windows-specific header definitions. Conclusion: Monkey Patching beats a Million #ifdefs To port LibreSSL to Windows, I essentially monkey patched Windows into both working, and failing, more like a POSIX operating system. This probably won't satisfy a POSIX conformance test, but it does automatically catch problems like close vs closesocket and maps errors consistently. The code does not need to deal with 2 different error systems, and any differences are fixed up one time in the compatibility layer. More importantly, it does this mostly through header and preprocessor magic, without adding a heavy-weight layer like Cygwin or needing to pepper the source code with #ifdefs. Having recently found some error handling, #ifdef-related bugs in Meterpreter, it will be interesting to apply similar portability techniques there as well.

Featured Research

National Exposure Index 2017

The National Exposure Index is an exploration of data derived from Project Sonar, Rapid7's security research project that gains insights into global exposure to common vulnerabilities through internet-wide surveys.

Learn More

Upcoming Event


Rapid7's annual security summit is taking place September 12-14, 2017 in Boston, MA. Join industry peers for candid talks, focused trainings, and roundtable discussions that accelerate innovation, reduce risk, and advance your business.

Register Now


Security Nation

Security Nation is a podcast dedicated to covering all things infosec – from what's making headlines to practical tips for organizations looking to improve their own security programs. Host Kyle Flaherty has been knee–deep in the security sector for nearly two decades. At Rapid7 he leads a solutions-focused team with the mission of helping security professionals do their jobs.

Listen Now