Sysdig Falco is a new open source endpoint security monitoring tool for Linux released by Sysdig. It allows you to create simple rules to alert on behaviors you find suspicious. Sysdig falco also supports containers, so you can easily alert on container activity.
Deploying Sysdig Falco
Sysdig Falco deploys on Linux servers. Sysdig provides several mechanisms for deploying, including a one-liner
sudo bash script and package management repos.
To get started, you can deploy Sysdig using a bash one-liner. Note, this method is useful for testing and experimentation, but due to security concerns you should probably not be using this method for production deployment.
# curl -s https://s3.amazonaws.com/download.draios.com/stable/install-falco | sudo bash
- Deployment note: Sysdig Falco requires deployment of a kernel module to your host servers, so make sure confirm this is ok with your deployment policies, and are whitelisting the behavior (if you are doing any monitoring or restrictions of kernel modules).
Let’s try running
falco to confirm everything was installed correctly. Passing the
–help flag indicates several options for launching
# falco --help Usage: falco [options] Options: -h, --help Print this page -c Configuration file (default /mnt/workspace/falco-build-stable/label/builder-agent-64/falco/falco.yaml, /etc/falco.yaml) -o, --option <key>=<val> Set the value of option <key> to <val>. Overrides values in configuration file. <key> can be a two-part <key>.<subkey> -d, --daemon Run as a daemon -p, --pidfile <pid_file> When run as a daemon, write pid to specified file -e <events_file> Read the events from <events_file> (in .scap format) instead of tapping into live. -r <rules_file> Rules file (defaults to value set in configuration file, or /etc/falco_rules.yaml). -L Show the name and description of all rules and exit. -l <rule> Show the name and description of the rule with name <rule> and exit.
Sysdig Falco looks for its configuration using two yaml files which are installed by default in
The paths can be dynamically configured using the flags
falco -c <config file> -r <rules file>
Let’s look at /etc/falco.yaml, which controls several logging and high-level configuration items:
# cat /etc/falco.yaml # File containing Falco rules, loaded at startup. rules_file: /etc/falco_rules.yaml # Whether to output events in json or text json_output: false # Send information logs to stderr and/or syslog Note these are *not* security # notification logs! These are just Falco lifecycle (and possibly error) logs. log_stderr: true log_syslog: true # Where security notifications should go. # Multiple outputs can be enabled. syslog_output: enabled: true file_output: enabled: false filename: ./events.txt stdout_output: enabled: true
As you can see, there are several options for configuring security event output (to a file, syslog, or stdout) along with two options for formats (text vs JSON). Let’s create a rule, and look at the example output for both.
Creating A Simple Alert Rule
Sysdig Falco comes with a simple rules filtering language for defining alerts. Using this mechanism, you can alert on a number of things regarding behavior on a system (or ‘inside’ of a container, as we’ll look at below). The rules by default are stored in /etc/falco_rules.yaml, and Sysdig has provided some default rules for you to use.
look for a rule for
Let’s modify that rule slightly to alert on any and all network traffic:
- rule: system_binaries_network_activity desc: any network activity performed by system binaries that are not expected to send or receive any network traffic condition: ((inbound or outbound) and (fd.sockfamily = ip)) and fd.name != '' output: "Suspicious binary sent/received network traffic (user=%user.name command=%proc.cmdline connection=%fd.name type=%evt.type)" priority: WARNING
As you can see, there are a few elements to building a rule. Let’s look at these fields in detail.
- rule: the identifier of the rule.
- desc: a description of the rule, e.g. “rule for alerting on network traffic”
- condition: a rule written in the simple Sysdig filtering language
- output: The output message for the rule in Sysdig’s output formatting. Note: you can use fields from the event which will get interpolated (see http://www.sysdig.org/wiki/sysdig-user-guide/#output-formatting)
Now, let’s try running
# falco Mon Jun 6 06:46:16 2016: Falco initialized with configuration file /etc/falco.yaml Mon Jun 6 06:46:16 2016: Parsed rules from file /etc/falco_rules.yaml
Try pinging a hostname, or opening a browser and going to a website. You’ll see right away alert messages being logged to stdout. By default, they should be in the text format, but you can try switching the
json_output: true in
/etc/falco.yaml to see examples of the JSON format. I have included examples of both below:
Text log example:
06:29:38.911653567: Warning Suspicious binary sent/received network traffic (user=root command=Chrome_IOThread connection=10.0.1.10:34165->18.104.22.168:443 type=connect)
JSON log example:
- The JSON format is nice for parsing, but at the current time, seems to lose some of the ‘alerting’ information you’d like (e.g. the output message).
ruleidentifier is unfortunately not included in the alert message itself.
- The priority defined in the rule is not included in the message itself (but is used by the syslog facility)
Sysdig Falco can also be used to monitor activity inside of containers. The recommended deployment strategy is to deploy on the host of the container, which will allow for full visibility throughout all of the containers running on the host – although some limited functionality can be gleaned by running Sysdig Falco inside a container itself.
Let’s look at a simple alert rule specific to some container activity to see the container monitoring in action!
First, in one terminal window, let’s pull the container we are going to used to test, and run it:
# docker pull ubuntu:14.04 # docker run --rm -it ubuntu:14.04 /bin/bash $
You should have a bash prompt inside of the container which we can use for testing.
On the host, let’s also modify the rule we created in
/etc/falco_rules.yml to look for network activity **inside of a container **and record the the container id in our output message:
- rule: system_binaries_network_activity_container desc: any network activity performed by system binaries that are not expected to send or receive any network traffic in a container condition: ((inbound or outbound) and (fd.sockfamily = ip)) and fd.name != '' and container output: "Suspicious binary sent/received network traffic from container=%container.id (user=%user.name command=%proc.cmdlin e connection=%fd.name type=%evt.type)" priority: WARNING
Now let’s try running falco, and then typing
ping google.com in our container!
# falco Mon Jun 6 06:57:56 2016: Falco initialized with configuration file /etc/falco.yaml Mon Jun 6 06:57:56 2016: Parsed rules from file /etc/falco_rules.yaml 06:57:58.143196226: Warning Suspicious binary sent/received network traffic from container=9a96cb629b9a (user=root command=ping google.com connection=172.17.0.2:54523->22.214.171.124:53 type=connect) 06:57:58.177714266: Warning Suspicious binary sent/received network traffic from container=9a96cb629b9a (user=root command=ping google.com connection=172.17.0.2:34597->126.96.36.199:1025 type=connect) 06:57:58.196316749: Warning Suspicious binary sent/received network traffic from container=9a96cb629b9a (user=root command=ping google.com connection=172.17.0.2:38879->188.8.131.52:53 type=connect)
You should now see the container activity, along with the container id!
Sysdig Falco appears to be a promising new tool in a defender’s toolkit, but there are some limitations you should be aware of as you are planning your deployments.
No centralized collection facilities. Sysdig Falco is only an endpoint tool; it does not come with integrations for any centralized collection or aggregation facilities. You can integrate it with Syslog or collect the individual logs from files as described above, and send to your Splunk, Logstash, or SIEM, but this will require some programming lift for your security engineers.
No built-in “signature management.” Signatures for Sysdig Falco are stored in a file, which again lacks any centralized management so to deploy any signature changes, you must integrate with a local configuration management system (e.g. Chef, Puppet, Ansible) to manage updates to these files.
No event aggregation. As observed above, often, you’ll get multiple events that are really related to the same activity. To avoid ‘alert fatigue’ it would be nice if these events are aggregated into a single alert for viewing/testing purposes, but right now you must accomplish this within a SIEM or log management system.
Beware of any potential performance hit. Sysdig works by using a kernel module to hook various kernel activity so beware of any potential performance hit on a system that requires high-performance, such as a distributed database. The product is mature and has undergone a lot of testing, but as with any system that is introducing a lot of logging, it’s probably a good idea to do some benchmarks using the rulesets you’d like to deploy before you deploy Sysdig Falco on a machine where performance is critical.