Virtual Machine Automation (vm-automation) repository released

Last updated at Mon, 21 Aug 2017 19:02:41 GMT

Rapid7 just released a new public repo called vm-automation. The vm-automation repository is a Python library that encapsulates existing methodologies for virtual machine and hypervisor automation and provides a platform-agnostic Python API. Currently, only ESXi and VMWare workstation are supported, but I have high hopes we will support other hypervisors in time, and we would love to see contributors come forward and assist in supporting them!

That's awesome. I want to get started now!

Great! Instructions on how to use the library are here: https://github.com/rapid7/vm-automation

Why?

The Metasploit team has an embarrassment of riches when it comes to modules and payloads thanks to our amazing community and staff. To give some idea of the embarrassment of riches, feel free to launch msfconsole and check the output:

       =[ metasploit v4.15.0-dev-7e1b50a                  ]
+ -- --=[ 1665 exploits - 953 auxiliary - 294 post        ]
+ -- --=[ 486 payloads - 40 encoders - 9 nops             ]
+ -- --=[ Free Metasploit Pro trial: http://r-7.co/trymsp ]

We have 486 payloads, 1,665 exploits, nearly 1,000 aux modules, and 294 post modules. Additionally, we have 443 super-awesome contributors across the globe sending us modules every single day. All this is impressive, and we are incredibly thankful for everyone's support. At the same time, this is a challenge to test—especially since Microsoft and Linux keep updating things to break our code without warning (don't they know who we are?!).

One of the efforts that we are working on is some test automation to help us maintain our modules and payloads—or at least know when things break faster—and to streamline the PR landing process. To do that we made a testing infrastructure that uses virtual and physical machines as attackers and targets; then we launch payloads, scripts, and modules on the virtual machines and track the responses. As we are all lazy, it needed automation, so we looked for a clean, simple way to interact with different kinds of vms that was consistent across hypervisors. In a former life, I was also an instructor and CTF developer; as a result, I know that ability to script vm management tasks makes life much easier for a lot of people beyond the narrow case of module and payload testing in Metasploit, so we split the library for automating vm tasks into a separate repo for anyone to use (and contribute new ideas!).

Aren't there already things that do this?

Yes...sort of. There are multiple projects out there that exist and give varying amounts of control over vms using lots of different languages. Pyvmomi is one great example; it allows spectacular levels of customization and power over virtual machines that the average CTF-er or tester has absolutely no need to use, while simple tasks like getting a list of snapshots take ~40 lines of code. I certainly do not want to denigrate or disparage Pyvmomi: they provide an awesome API, and I know people who need that level of power over virtual machines, but it is just too powerful and complex for a lot of hobby-level hypervisor scripters. This library wraps a lot of Pyvmomi API calls into simple, comprehensible API calls to support the majority of what most hypervisor script users would need, while abstracting a lot of the complexities in Pyvmomi.

Also, Pyvmomi only supports ESXi, and this library leverages Pyvmomi API calls to support ESXi, but then uses VMrun.exe to support VMware workstation. So while much of the underlying code is changing, the functions to interact with vms remain the same across hypervisors, supporting the main goal for this repo: one function call, multiple hypervisors.

So what is it you say you do around here?

The supported functions are currently limited to those you might want to automate a CTF or test-range:

• checkTools
  • Returns the state of VMWare tools
• deleteSnapshot
  • Deletes a given snapshot
• getArch
  • Returns the vm architecture
• getFileFromGuest
  • Pulls a file from the virtual machine
• getSnapshots
  • Updates the vm object's snapshot list attribute
• getVmIp
  • Updates the vm object's IP address to match the vm
• getUsername
  • Returns the vm's username
• isPoweredOff
  • Returns true or false
• isPoweredOn
  • Returns true or false
• makeDirOnGuest
  • Creates a directory on the specified vm
• powerOn
  • Turns on the vm
• powerOff
  • Turns off the vm
• revertToSnapshot
  • Reverts the vm to a given snapshot
• runCmdOnGuest
  • Runs a command or executable on the vm
• setPassword
  • Updates the password in the vm object
• setUsername
  • Updates the username in the vm object
• takeSnapshot
  • Takes a snapshot of the vm
• updateProcList
  • Updates the process list in the vm object
• uploadAndRun
  • Uploads a script or executable file and runs it
• uploadFileToGuest
  • Uploads a file to the vm
• waitForTask
  • Waits for a given task to complete before allowing continued execution. Most of the API calls can be synchronous or asynchronous. This function allows us to toggle between the two.

How are you implementing the functions?

The basic layout is this: for each hypervisor (currently two), there are two classes. The first class is the hypervisor class. It contains all the attributes required to make the hypervisor work, like IP address, login information, and vm list. The other class is the vm class with supporting functions and attributes associated with the vms to handle normal vm interactions with snapshots, process lists, IP addresses, and the hypervisor. By overloading the function names across the vm classes, we can interact with any vm exactly the same, regardless of the hypervisor (or type of hypervisor) on which it runs.

Moving forward

The obvious thing is that we need to support more hypervisors: I would love to support cheaper or free virtualization options like VirtualBox or even Hyper-V.

I hope that this library proves as useful to others as it would have been to me over the years. I welcome anyone who would like to contribute, especially if they want to start work on supporting extra hypervisors! It is a relatively simple project. I think if we do it right it will see a lot of use, and we can help a lot of people.