We recently released our integration with New Relic, a leading Application Performance Monitoring (APM) solution – we’re really excited about showing our customers how offline AVM impacts their application performance online. We wrote the blog post for their blog, and shared more about our integration with New Relic and the benefits this partnership can bring to our customers.

When designing an application, there’s often a trade-off between performance and security. For example, using a larger key size for encryption will make it harder to decipher, but it will also take more resources to encrypt and decrypt. When it comes to writing secure source code, however, this performance/security trade-off may not apply.

Application vulnerability mitigation (AVM) solutions, such as Lucent Sky AVM, are designed to help secure applications by automatically identifying vulnerable pieces of source code and replacing them with secured ones. Imagine something like spell check, which both finds and offers a fix for mistakes (in our case, things like cross-site scripting and SQL injection). Because Lucent Sky AVM (like traditional Static Application Security Testing, or SAST, tools) works offline, it’s not immediately apparent what impact these vulnerability fixes have on source code come run-time. Developers and IT professionals will often ask if the security benefits of using AVM technology will reduce the application’s performance like WAF and RASP do. That’s where Lucent Sky’s integration with New Relic comes in.

The integration between New Relic APM and Lucent Sky AVM is fairly simple: Users set up their New Relic account information in the Lucent Sky AVM server, and applications on the server are linked with those in New Relic. By using New Relic APM with Lucent Sky AVM, users not only gain performance insights on their applications, but can also monitor how the vulnerability mitigation affects application performance.

Remediated source code profiled

To demonstrate how linking Lucent Sky AVM and New Relic APM can help customers better see the link between AVM technology and application performance, we tried the integration on a 3-tier ASP.NET application with around 220,000 lines of code. Multiple instances of Apache Bench were used to simulate 50 concurrent requests. We used New Relic APM to monitor the time it took for the application to process each request, while Apache Bench itself monitored the performance from the client’s perspective.

Here’s a snapshot of the original code:

And after remediated by Lucent Sky AVM:

Inside the .NET CLR, it took the original application 1.13ms to handle each request, while the secured application used 1.18ms. That’s about 4.4% slower.

Now let’s take a look from a website visitor’s perspective. Here’s a scatter chart of the response time recorded by the test client:

With the original application (grey dots), the average response time is 17.65ms, with 50% of the requests having a response time of 15ms and 99% of requests being completed within 31ms. The secured application (blue dots) has a slightly higher average response time of 18.75ms, with the same response time distribution: 50% of the requests have a response time of 15ms, and 99% of the requests were completed within 31ms. A website user won’t notice any performance difference (unless she’s the lucky 1%).

The put the numbers in perspective, we installed open source web application firewall ModSecurity on the server running the original code, enabling the baseline rules from OWASP’s Core Rule Set.

With ModSecurity installed and baseline rules enabled (orange dots), the web server spent an average of 36.98ms to complete each request, with 50% of the requests having a response time of 31ms and 99% of the requests being completed within 47ms. Although comparing AVM secured code with ModSecurity isn’t really apples to apples, it does show how little performance impact AVM secured code has compared to host-based web application firewalls.

The goal of Lucent Sky AVM is to provide functional actions and insight to developers, resulting in more efficient and secure code releases. By integrating with New Relic APM we can see the specific performance results of implementing automatic fixes to secure source code, and can benchmark AVM secured code against other solutions in run-time.

We’ll continue to test our own and client applications with New Relic, and look forward to seeing how better tools and processes offline can continue to improve run-time results.