Installing Open-appsec for Linux on Nginx

As ModSecurity approaches its end of life in July 2024, many organizations are looking for a modern alternative to secure their web applications and APIs. Open-appsec is positioning itself as a leading open-source option for replacing ModSecurity. Open-appsec is an open-source fully automated Web Application & API Security solution by Checkpoint.

Powered by machine learning, Open-appsec provides a fully automated approach to web application and API protection. Rather than relying on static signatures like ModSecurity, Open-appsec continuously analyzes network traffic to proactively detect both known and zero-day threats. This provides stronger protections against today's rapidly evolving cyberattacks. Here is a comparison between NGINX App Protect, ModSecurity WAF, and open-appsec:

Open-appsec can be deployed as an add-on to common platforms like NGINX, Kubernetes Ingress Controller, and Kong API Gateway. It claims to offer simpler maintenance than signature-based WAFs through automated learning and exception handling. With no need for manual rule tuning, Open-appsec aims to free up security resources.

In this blog post, we are going to deploy open-appsec for Linux on Nginx and test its capabilities to see how well it can protect our web applications.

Prerequisites

To install open-appsec, we are going to need the following:

Linux machine with:
- A supported OS and NGINX or Kong version. A list of all supported/pre-compiled attachments for NGINX and Kong per supported OS versions is available here.
- In case your versions are not supported yet, you can also build the code yourself, see here.
- Root permissions
wget command-line tool installed on your Linux machine

Installation

Download the installer for Linux using these commands:

wget https://downloads.openappsec.io/open-appsec-install && chmod +x open-appsec-install

You can show the installer version and available options by running the following command to show the help info:

./open-appsec-install -h

This interactive installer provides 2 alternative modes for automatic vs. manual installation:

Mode 1: Automatic installation of Open-appsec and adding attachment (plugin) to NGINX

In this mode, open-appsec will automatically be installed with all required components and the attachment will be added and activated in the existing configuration for NGINX/Kong.

./open-appsec-install --auto

Optional open-appsec installer parameters

--token allows connecting directly to SaaS management. We will be adding the agent later to the management portal, and so you can skip this.
--prevent will set the default rule in the default policy file to prevent-learn instead of detect-learn, but the recommendation is to keep detect-learn as the default rule.

Mode 2: Download of software components and presenting manual installation instructions

In this mode all required components based on your NGINX version, OS version, Platform will be downloaded to your machine and instructions are presented for manual installation.

./open-appsec-install --download

Optionally, you can add a --tmpdir <path> option to specify an alternative path for the downloaded software components (default path is /tmp/open-appsec/)

Once the download has finished, follow these steps for manual installation:

Step 1: Deploying the attachment on an existing alpine NGINX/Kong server

Copy the associated libraries as shown in the output for Step 1 with commands similar to this:

cp /tmp/open-appsec/[version specific dir]/libosrc_shmem_ipc.so /usr/lib/libosrc_shmem_ipc.so
cp /tmp/open-appsec/[version specific dir]/libosrc_compression_utils.so /usr/lib/libosrc_compression_utils.so
cp /tmp/open-appsec/[version specific dir]/libosrc_nginx_attachment_util.so /usr/lib/libosrc_nginx_attachment_util.so

Copy the nginx attachment file as shown in the output for Step 1 with command similar to this:

cp /tmp/open-appsec/[version specific dir]/ngx_cp_attachment_module.so /usr/lib/nginx/modules/ngx_cp_attachment_module.so

Load the attachment on your NGINX by adding the following line to your nginx.conf, usually located here: /etc/nginx/

load_module /usr/lib/nginx/modules/ngx_cp_attachment_module.so;

Step 2: Installing open-appsec agent

Run the following commands as shown in the output

/tmp/open-appsec/openappsec/install-cp-nano-agent.sh --install --hybrid_mode --server 'NGINX Server'
/tmp/open-appsec/openappsec/install-cp-nano-service-http-transaction-handler.sh --install

Step 3 Validate configuration

nginx -t
service nginx restart

How to use the open-appsec-ctl Tool

The interactive CLI tool open-appsec-ctl allows you to perform various tasks related to your open-appsec for NGINX/Kong installation. The tool will be automatically installed with the agent. To check the available options that can be used with the tool, run:

open-appsec-ctl -h

To list available policies, run open-appsec-ctl --list-policies or open-appsec-ctl -lp. Currently, only a single configuration file is supported, support for multiple configuration files will be added soon.

You can view, edit, apply policies and event manage the open-appsec agent using the open-appsec-ctl tool. For a full list of available options, check out their official documentation here.

Configuring open-appsec using Local Policy File

When using open-appsec for NGINX on Linux, the configuration can be done in a declarative way using a single YAML file which holds all the relevant configuration objects. This way it is fully compatible for integration in GitOps CD-based processes. Alternatively, management can also soon be done centrally via the open-appsec Web UI.

The default location of the declarative configuration file is /etc/cp/conf/open-appsec.yaml. You can show and edit the full default declarative configuration file using your default text editor like nano or vim or by using the interactive CLI tool open-appsec-ctl --edit-policy [policy-file].

The default policy within the default configuration file, which is created during the installation, contains the following setting which sets the mode to detect-learn for all web resources provided by the NGINX:

policies.default.mode: detect-learn

If you want attacks instead to be prevented by default for all web resources, you can change this as follows:

policies.default.mode: prevent-learn

The section policies.specific-rules allows you to create specific rule entries for specific hostnames, hostname-path combinations or paths which override the default policy.

It is recommended that once sufficient confidence was gained in detect-learn mode that you add specific rules in the policies.specific-rules section for those hostnames, hostname-path combinations or paths for which you want to switch to prevent-learn mode

You find the full specification for the different configuration sections and elements that can be part of the declarative configuration file, including an example for each, in the Advanced policy configuration documentation

Monitoring events

To monitor events, you can use the following command:

open-appsec-ctl --view-logs

By default, logs are stored here: /var/log/nano_agent/cp-nano-http-transaction-handler.log<number>

You can configure flexible logging according to your requirements by using a custom log-trigger. You can find details of how to configure the log trigger when using declarative configuration file in the Advanced policy configuration documentation

You can also view the logs on the web UI, as discussed below.

Using open-appsec with the Web UI

Open-appsec provides a cloud-hosted central management for assets and policies, cloud logging, graphical dashboards, events analysis and ability to manage multiple deployments/clusters in a scalable way.

You can find full details on how to use the Web UI on the official documentation

How Good is Open-appsec WAF?

There is no point of setting up a WAF if it does not do what it is supposed to do - protect the web resources. To test open-appsec strength, I used the waf-bypass tool by nemesida. I ran this tool on a website running WordPress 6.4.1 on Nginx web server. You can check out how to install WordPress with Nginx in this guide.

WAF bypass Tool is an open source tool to analyze the security of any WAF for False Positives and False Negatives using predefined and customizable payloads. Check your WAF before an attacker does. WAF Bypass Tool is developed by Nemesida WAF team with the participation of community.

💡

Note: - This is by NO means a comprehensive test on how good/bad open-appsec actually is. However, testing it side by side with other well known WAFs gives us a good indicator on how it might perform against its competitors

Running the waf-bypass tool while having open-appsec installed, gave the following result:

Running the same waf-bypass tool on the same website but with Cloudflare (free version) as the WAF, gave the following results:

To give as a different perspective of how well Open-appsec performs against other WAFs, the open-appsec team also ran some results, comparing the different WAFS in the market these were the results:

Of course, we can take this results with a pinch of salt, since the tests were done with the open-appsec team, and it could be a way of marketing the product, but it gives us a rough idea of how the WAF can perform against the competitors. Here is the link to the full comparison.

We will be doing a full test soon of this and may other WAFs so make sure you subscribe to our newsletter and get notified once the results are out.

Conclusion

Open-appsec is a solid option, especially for people looking to move away from ModSecurity but still want a flexible and open-souce WAF. The machine learning capabilities give it an edge over other signature based WAFs, and it has the potential of blocking zero-day attacks, as they claim. With a good backing of Checkpoint, a leader in the IT security space, and community contributions, I believe open-appsec is a good WAF. Feel free to check it out and leave your comments below. I will also be making follow-up tutorials on the same with more advanced options, so feel free to subscribe to the newsletter so that you don't miss out on that.

On this page