Yesterday we did an impromptu (completely unrehearsed) live Q&A titled ‘The Heartbleed War Room Webcast' which you can go listen to here: http://information.rapid7.com/heartbleed-war-room.html

On this webcast we had

  • Trey Ford, Rapid7's Global Security Strategist (@treyford)
  • Mark Schloesser, Security Researcher at Rapid7 (@epmovsb), and
  • Josh Feinblum, Rapid7's VP of Information Security (@TheCustos)


For more information, please visit our resource page:


We were unable to answer everyone's questions on the call, so we've answered most of them here. (There was a question or two held back to build blog posts on - some of these questions will do that!)
There were a number of questions relating to Metasploit and Nexpose, we will have those posted in the next 24 hours. Thanks again for joining us - if you have additional questions, feel free put them in the comments below!

  • My application makes SSL connections using OpenSSL with other hosts. I wonder about updating the clients, not only the servers. Can you elaborate on that?
    • SSL/TLS is a protocol that secures communication - so needs to be supported by both ends of the channel. Clients such as Browsers but also desktop applications, browser extensions - they all use SSL and most likely OpenSSL. If they use a vulnerable version, they need to be updated. On most Linux systems the clients/browsers use the system OpenSSL version so that only the system library version needs to be updated.
  • Can  you please demo this exploit please?
    • Technically the exploit establishes a partial SSL connection and then sends a heartbeat request with a crafted length field. The response from vulnerable OpenSSL versions contains unsanitized memory content. This can be done with the metasploit module, other proof-of-concept scripts or website checks like http://filippo.io/Heartbleed/
  • Is this attack detectable?
  • To understand further, OpenSSL 1.0.0 branch and 0.9.8 are not vulnerable to this particular attack but I would guess just as vulnerable to other ssl attacks?
    • As far as we're aware these older versions contain no disclosed remote vulnerabilities. The code responsible for the heartbleed bug was introduced first with 1.0.1 in March 2012. Thus any systems using older versions are considered secure at this point.
  • Why is CVSS for CVE-2014-0160 only rated as medium (5.0)?
  • Q: What client side browsers use openssl that are impacted?
  • Q: Have you seen this being exploited in the wild?
    • Since the public disclosure we've seen several checks and exploits being run against our infrastructure. It is unclear which of these are just users trying to protect themselves, researchers or actual malicious data exfiltration attempts.
  • How can I perform forensics in a hearbleed attack situation?
    • Unless you have full packet captures it is hard to determine which data was leaked. We advise to consider credentials and keying material as leaked on affected systems.
  • How can I check for heartbleed during a pentest?
    • You can use public PoCs or the Metasploit modules to check for the vulnerability.
  • What are the tools or scripts to test heartbleed?
  • Which all servers are vulnerable ?
    • Web servers, mail servers, databases, chat services, some SSL VPNs - anything using affected SSL versions.
  • What kind of issue is this on devices that are not internet facing?
    • Obviously the exposure level is lower on an ‘internal network' than Internet facing services - so they should be addressed after higher risk assets have been addressed.
    • Being on an internal segment does not remove the requirement to get this fixed - any place you have deployed OpenSSL is in scope, you need get it patched.
    • Compromised workstations / laptops or misbehaving insiders can be used to leverage this vulnerability to gather data including, but not limited to cookies, session data, usernames and passwords, as well as private key material.
  • What skillset is required to use this vulnerability?
    • As the vulnerability is relatively simple, there are dozens of test scripts already and thus it is a matter of using any of these to leak memory from vulnerable SSL endpoints. Interpreting the leaked data can be more challenging - and sometimes there might not even be any useful data in there.
  • In regards to most people not being able to look back and see if they were compromised.  I assume you are referring to pcaps as I recall this particular vector is not logged in any sort of default enabled logs for the service(s) using ssl?
    • Exactly, if you have full network captures, then it might be possible to check for exploitation / compromise. There is no explicit logging of this feature.
  • Should we ask our users to change their passwords both on vulnerable but patched and also on non-vulnerable systems?
    • If a certain vendor / system was not vulnerable, then there should be no issue - but that might be hard to determine unless they specifically say so. In general we would advise to just use this as a chance to update passwords - which should be done regularly anyway.
  • There seems to be different sides in password resetting if you should or should not immediately change your password.  What should we tell our users who use Yahoo, Google, etc. for web or mail?
    • If the respective service provider is patched / updated, then changing passwords is advised. If the status is unclear, please press the provider to acknowledge the event and communicate clearly when they have patched, and if they advise password resets.
    • I would steer clear of sites not communicating this clearly.
    • This may be an ideal time to evaluate moving to two factor authentication, learn more here.
  • Is the MSF module used to test this vulnerability in community edition or just Pro?
    • Always in both. There are no vulnerability checks / exploits in Pro that are not in framework / community.
  • Can you talk about PFS? (Perfect Forward Secrecy) If enabled, are you still vulnerable completely or even partly?
    • Still vulnerable, however PFS protects against decryption of previous communication in case the keying material is compromised.
  • Do you think this will promote the use of Perfect Forward Secrecy?
    • PFS unrelated to Heartbleed. But PFS certainly is a good idea to have.
    • We hope to see more organizations enabling PFS during their mitigation workflows.
  • We have star cert for many systems, often called a wildcard certificate. If one system is vulnerable and at risk that uses the star cert, does it make the star cert vulnerable on all servers that use the cert?
    • Yes, there needs to be a new key and new wildcard cert issued and the old one replaced on all systems. The key might not have been compromised but we still advise that you deploy a new one.
  • For web servers running Apache that uses openssl how can this be mitigated
    • In the same way as for anything else, update OpenSSL and renew SSL key and certificate.
  • What specific checks are being performed by the Nexpose vulnerability scanner in scanning for the existence of the OpenSSL vulnerability?
    • It will try to make the remote system create the too large response message that contains unsanitized memory contents. If it receives such a response, the system is considered to be vulnerable. The check minimizes the amount of memory leaked.
  • Can you provide again on step by step of what we should be doing from a compliance standpoint?
    • This can get into a religious discussion, so I will be direct - compliance should be a by-product of a good security program. Your after action documentation for the Heartbleed event (or any other event) should be kept on file.
  • Some said forcing password changes... is this only internet facing passwords?   Also, can we find compromised systems?  How fast do we need to get this fixed?
    • For Internet facing systems that you have fixed this vulnerability, I would urge you to discuss the pros and cons of forcing a password reset for your users- that is ultimately your management's decision. I would also urge you to always err on the side of protecting user safety.
  • ScanNow is giving an error that says it requires JRE 1.6.0.  1.6 and 1.7 installed.  ?
    • We are aware of an issue with 64 bit JRE versions. Please use a 32 bit JRE for now, either 1.6 or 1.7 should work.
  • What is the easiest and quickest way to scan for this internally for someone with no existing scanning infrastructure in place?
    • We released a ScanNow tool to do this - otherwise use Metasploit.
  • This is basically vulnerable because of the heartbeat. why not just turn off heartbeat and allow renegotiation?
    • Turning off the heartbeat feature is one mitigation if one can not update to the latest version, yes. However heartbeats are unrelated to renegotiation.
  • How long will the scan run to identify if we are vulnerable to heartbleed?
    • Depends on network / target range size.
  • Putting Edward Snowden Hat on: Any opinion if this code insertion was intentional or unintended?
    • Conspiracy theory discussions are fun, and we will happily pontificate with you over a beer, but won't get into that here.
  • What is the client device risk of applications using vulnerable openssl, outside mutual ssl  authentication?
    • Regardless of mutual authentication, client applications can leak data in the same way as servers do. This might be a problem, depending on the application and memory content- let us know what you find.
  • Has there been any response or general concern around TLS email transmissions between companies and the likelihood of exploit of TLS servers and need to change those keys as well?
    • The threat is applicable in the same way as any other TLS communications. If systems are only reachable by trusted peers, then of course any misuse is unlikely.
  • If a vendor's web server has been patched, should we re-issue our client side certificate after it has been patched.
    • That is an interesting question - and certainly raises the cost of containment and cleanup. I would discuss this with your in-house red team, or contact us at info@rapid7.com to discuss this further.
  • The initial report was that this was only affecting beta versions, is that not correct?
    • No, several very popular stable versions are affected - 1.0.1 through 1.0.1f.
  • Can you please describe the technical details of how the vulnerability works?
    • Unsanitized input, concretely a length field, is used to allocate a memory area for the response which subsequently is only partially filled with data from the request. The rest of the memory area can contain any data and is sent over to the remote side with the response.
  • This vulnerability copies maximum of 64KB of server process memory. What do you think the chances are the server private keys of the server will reside in that 64kb range of process memory.
    • We don't have any statistics / numbers on the probability. But we have seen proof of private keys being leaked in certain situations. So even if for most systems it would be very unlikely, we still suggest to assume the worst and re-new the key and certificate. Better safe than sorry. 
  • ScanNow seems to be for UPNP. Does it also work to detect heartbleed?
  • With regards to testing servers for the heartbleed vuln, should we be cautioning people against using random websites that offer heartbleed checking. Seems like a fairly solid method for gathering intel on vulnerable sites, especially if not run be a reputable entity.
    • Absolutely correct, the people running these sites will have a decent set of vulnerable sites in their logs. So a level of trust should exist before using such a site. We suggest to use the proof-of-concept tools, Metasploit or our ScanNow tool.
  • Are SSH/SFTP connections affected by this vulnerability?
    • The SSH protocol does not use SSL/TLS - and is thus not vulnerable to the attack.
  • I hear mention of web servers, what about networking products, firewalls, and other systems that use OpenSSL..  Any word on those environments
    • If they make use of the vulnerable OpenSSL versions for their SSL / TLS connections, then it is highly likely that they are vulnerable.
  • Q: Can SSL VPN tunnels exploited by this ?
    • The OpenVPN team has issued updates and stated that the software is vulnerable if using the mentioned OpenSSL versions. We assume that for other SSL VPN solutions the same applies.
  • Has anyone reported actually being breached as a result of this at this point?
    • We have seen reports of credentials, session cookies and key material being leaked. It is unclear whether these were used for any malicious purpose.
  • What are the next steps after you patch the openSSL library? Certs, keys, passwords?
  • Do client systems like Windows 7 OS might have inbuilt modules that would use OpenSSL or similar implementations that might have similar defects?
    • Good question, we do not have that data yet.
  • What about Medical Devices and FDA certification?
    • Broad question, good line of thought. Any system running a vulnerable version of OpenSSL Vulnerable to the Heartbleed bug. Obviously systems that go through certification processes will be vulnerable until an updated certified release is issued.
    • We recommend that sensitive systems (think ATMs, Medical Devices, Point of Sale systems) be kept on isolated networks with strict access control, they will need additional protection and monitoring.
  • How is this vuln different to a typical buffer overflow? Wouldn't it be possible to run a shellcode by exploiting this vuln?
    • The vulnerability does not allow any remote code execution. It is purely an information / data leak bug.
  • What are you thoughts / recommendation on setting up a Honey Pot in my address space to monitor scans?
    • As mentioned in the webcast, we recommend this be done on internal network segments can be a good idea to pinpoint compromised systems or insider threats.
    • The signal-to-noise ratio of doing this outside the firewall may not be a fruitful investment of time and energy.
  • I may have missed this question being asked already but does it affect anything that OpenSSL signed or only HTTPS. Does it affect SMTP over TLS, FTPS, etc?
    • All software using SSL/TLS through vulnerable OpenSSL versions are affected equally.

As I said above, if you have additional questions, feel free put them in the comments below!