DEEP Measures secured its AI solution before launch – Netum’s security testing helped identify risks associated with new technology
As AI-powered applications become increasingly common, organizations are facing new cybersecurity challenges. Security testing of large language models requires expertise that differs from traditional penetration testing, with a stronger emphasis on understanding model behavior. Netum assessed the security of an AI chatbot developed by DEEP Measures, a software services provider for the pharmaceutical industry.
DEEP Measures is a Finnish software company whose online platform helps pharmaceutical companies leverage digital health technologies in clinical research. Netum had previously conducted a security assessment of the company’s web platform, but this was the first time it tested a service incorporating AI functionality. The objective was to ensure that the AI chatbot would be safe for customers to use.
According to Tony Riistaniemi, CTO of DEEP Measures, one of the biggest security risks associated with AI-based chatbots is the potential for misuse.
– Users may attempt to manipulate the chatbot in various ways, for example by steering it toward coding-related tasks, prompting it to reveal sensitive information, or using it for purposes it was never intended to support, says Riistaniemi.
Netum based its assessment on the OWASP Top 10 for LLM Applications framework and other AI security testing methodologies. A key aspect of the project was tailoring these practices to the customer’s specific solution. Joni Koskinen, the cybersecurity specialist responsible for the project, explains that AI application security testing is still a relatively new field that requires creativity and offers fewer established best practices than traditional penetration testing.
– Traditional penetration testing is highly technical in nature and follows well-established methodologies. AI application security testing, on the other hand, does not focus on analyzing network connections or encryption mechanisms. Instead, we interact directly with the chatbot through text-based prompts and attempt to manipulate or deceive it. One objective is to formulate inputs that reveal how the bot could potentially be misused. This type of work requires a great deal of imagination, says Koskinen.
In addition to technical expertise, testing AI applications requires the ability to creatively guide language models through different prompts and identify unexpected behaviour.
– Many customers are surprised by the number of ways a chatbot can be led off-topic and encouraged to perform tasks beyond its intended scope, Koskinen continues.
Continuity brings efficiency
At the end of the security assessment, the customer receives a comprehensive report that provides an overview of the key risks associated with the AI application and concrete recommendations for mitigating them. Tapani Höök, Cybersecurity Architect at Netum, points out that as AI applications become more widespread, organizations must recognize new categories of risks that traditional security testing alone does not address and that should be included as part of a comprehensive security testing strategy.
– In the case of DEEP Measures, the previously completed traditional security assessment provided a solid foundation for testing the AI application. This enabled the project team to move quickly and focus directly on the AI chatbot. The customer understands the value of long-term cybersecurity efforts. The project was also cost-effective because we did not have to start from scratch, says Höök.
Riistaniemi from DEEP Measures notes that the most positive surprise during the assessment was the absence of any critical findings. Alongside a few minor observations, the AI security testing provided valuable insights and raised important considerations for the future.
– AI is still a relatively new technology and has evolved so rapidly that the role of specialized experts is particularly important in security testing. Even with more than twenty years of experience in cybersecurity, I still find the behavior of language models somewhat mysterious. Netum’s AI security assessment helped us think through the different scenarios that could arise. I had several real ‘aha’ moments during the project, Riistaniemi explains.
Riistaniemi is highly satisfied with both the execution of the project and Netum’s expertise.
– I have no complaints whatsoever. Everything went exactly as agreed. In addition, the people at Netum are friendly and communication has always been smooth and straightforward, says Riistaniemi.
Netum likewise highlights the ease of collaboration. Koskinen praises the customer’s forward-thinking approach to improving cybersecurity.
– The customer has adopted exactly the right mindset toward cybersecurity: it is something worth investing in systematically and proactively rather than reacting too late. Continuity in maintaining and developing cybersecurity is absolutely essential.