Understanding Hidden Threats: Rootkits

Advances in Parallel Distributed Computing, 2011

International Journal of Information Security and Privacy, 2007

Respondents from eight Korean and US higher education institutions were surveyed as to their knowledge and experience with various forms of computer malware. The surveys provide insight into knowledge of rootkits that have become coffee lounge discussion following the once secretive Sony rootkit news break in late 2005 and then the rash of accusations and acknowledgments of other rootkits that followed. The surveys provide an empirical assessment of perceptions between students in the two countries with regard to various forms of malware. The two groups are similar in many respects, but they exhibit significant differences in self-reported perceptions of rootkit familiarity. US respondents report higher levels of familiarity for all assessed malware types, including the fictional “Trilobyte” virus. A timeline-based comparison between virus and rootkit knowledge reveals that relatively little is known about rootkits today. This highlights dangers related to existing knowledge levels ...

IFIP — The International Federation for Information Processing, 2007

Rootkits pose a dilemma in forensic investigations because hackers use them surreptitiously to mislead investigators. This paper analyzes the effectiveness of online and offline information analysis techniques in detecting rootkits and determining the processes and/or files hidden by rootkits. Five common rootkits were investigated using a live analysis tool, five rootkit detection tools (RDTs) and four offline analysis tools. The experimental results indicate that, while live analysis techniques provide a surprising amount of information and offline analysis provides accurate information, RDTs are the best approach for detecting rootkits and hidden processes.

2010

Rootkits refer to software that is used to hide the presence of malware from system/network administrators and permit an attacker to take control of a computer. In our previous work, we designed a system that would categorize rootkits based on the hooks that had been created. Focusing on rootkits that use inline function hooking techniques, we showed that our system could successfully categorize a sample of rootkits using unsupervised EM clustering. In this paper, we extend our previous work by outlining a new procedure to help system/network administrators identify the rootkits that have infected their machines. Using a logistic regression model for profiling families of rootkits, we were able to identify at least one of the rootkits that had infected each of the systems that we tested.

2010

Windows XP is the dominant operating system in the world today and rootkits have been a major concern for XP users. This paper provides an in-depth analysis of the rootkits that target that operating system, while focusing on those that use various hooking techniques to hide malware on a machine. We identify some of the weaknesses in the Windows XP architecture that rootkits exploit and then evaluate some of the anti-rootkit security features that Microsoft has unveiled in Vista and 7. To reduce the number of rootkit infections in the future, we suggest that Microsoft should take full advantage of Intel's four distinct privilege levels.

2007

A rootkit is a small, hard to detect computer program that stealthily invades an operating system or kernel and takes control of the computer. The rootkit can be placed on a computer by a hacker that gains unauthorized access to a computer, or by an unsuspecting authorized user that allows a virus or other malicious software to insert the rootkit into their computer. Cyberspace is full of threats and risks. Each danger must be carefully considered and protected against only to the extent that is reasonable and affordable in accordance with a prudent risk management program. When considering rootkits a risk manager will ask: How common are they? How severe are the consequences? How can they be prevented? How can they be removed? These general questions have been explored in a number of research projects and publications. At a finer level of detail and on a recurring basis, information assurance managers will also ask 'as of right now': How hard are they to create? How available is rootkit source code? How hard are they to install and operate? This paper describes a research project at Murray State University in which faculty and senior undergraduate students explored this second set of more time-sensitive questions. It describes both the pedagogical and technical issues of having students find rootkit source code on the web; getting the source code to run and operate in an academic laboratory without threatening the university's IT environment; and exploring what tools and techniques are currently available for detecting and removing rootkits.

Computer, 2006

CYBERSEC, 2016

Extended Abstract In the realm of cybersecurity, rootkits pose a credible threat to individuals, corporations, and governments. Through various techniques, rootkits are not only able to infect computer systems, but often times are able to remain undetected within a host for an extended amount of time by manipulating the system software, processes, libraries, or kernel. In this paper we give a general review of how rootkits operate, how they are employed in various cyberattacks, and the methods used for their detection. We consider specifically the effectiveness of detecting rootkits based on observable behavior such as system call timing variation. We compare the effectiveness of two neural network analysis approaches and a traditional linear statistical method for detecting the presence of rootkits. We report results of a case study analysis using the KBeast rootkit and our results indicate that neural networks are potentially more effective for detection methods in practice. The study also provides a foundation for further research on larger families of rootkits that alter system calls as part of their operational profile. Background The term rootkit refers to a program that has root privileges on a computer, along with the tools it needs to execute, but is mostly used in a negative context because of the cyber-attacks that utilize them. The complexity and use of rootkits has increased over time. Between 2001 and 2005, the use of rootkits on Windows based systems increased by 2300 percent, and from 2000 to 2005, the complexity of rootkits grew by 400% [1]. In 2004, of the 15,000 trojans reported by McAfee, 87% were rootkits [1]. A widely used method to classify rootkits was proposed by Joanna Rutkowska [2] that uses taxonomy of four categories, 0-III, based on rootkit behavior. Type I and II rootkits take steps to conceal their presence and differ based on what part of the operating system resources they affect, and we focus particularly on these types because of their ability to hide from detection. There are many methods for dealing with or identifying rootkits, including behavioral detection, integrity detection, and signature based detection [3]. We focus specifically on behavioral techniques that analyze how a system operates and tries to identify any deviations. Motivation and Case Study

2011

In monolithic operating systems, the kernel is the piece of code that executes with the highest privileges and has control over all the software running on a host. A successful attack against an operating system's kernel means a total and complete compromise of the running system. These attacks usually end with the installation of a rootkit, a stealthy piece of software running with kernel privileges. When a rootkit is present, no guarantees can be made about the correctness, privacy or isolation of the operating system.

2009

Traditional approaches to rootkit detection [45] [34] assume the execution of code at a privilege level below that of the operating system kernel, with the use of virtual machine technologies to enable the detection system itself to be immune from the virus or rootkit code. In this thesis, we approach the problem of rootkit detection from the standpoint of tracing and instrumentation techniques, which work from within the kernel and also modify the kernel's run-time state to detect aberrant control flows. We wish to investigate the role of emerging tracing frameworks (Kprobes, DTrace etc.) in enforcing operating system security without the reliance on a fullblown virtual machine just for the purposes of such policing. We first build a novel rootkit prototype that uses pattern-searching techniques to hijack hooks embedded in dynamically allocated memory, which we present as a showcase of emerging attack techniques. We then build an intrusion detection system -autoscopy, atop kprobes, that detects anomalous control flow patterns typically exhibited by rootkits within a running kernel. Furthermore, to validate our approach, we show that we were able to successfully detect 15 existing Linux rootkits. We also conduct performance analyses, which show the overhead of our system to range from 2% to 5% on a wide range of standard benchmarks. Thus by leveraging tracing frameworks within operating systems, we show that it is possible to introduce real-world security in devices where performance and resource constraints are tantamount to security considerations.