Application Intrusion Detection using Language Library Calls

Traditionally, intrusion detection systems detect intrusions at the operating system (OS) level. In this paper we explore the possibility of detecting intrusion at the application level by using rich application semantics. We use short sequences of language library calls as signatures. We consider library call signatures to be more application-oriented than system call signatures because they are a more direct reflection of application code. Most applications are written in a higher-level language with an associated support library, such as C or C++. We hypothesize that library call signatures can be used to detect attacks that cause perturbation in the application code. We are hopeful that this technique will be amenable to detecting attacks that are carried out by internal intruders, who are viewed as legitimate users by an operating system.
Note: Abstract extracted from PDF text

Jones, Anita, and Yu Lin. "Application Intrusion Detection using Language Library Calls." University of Virginia Dept. of Computer Science Tech Report (2001).