Skip navigation.
Home

dannyquist's blog

New Search System, No More Accounts Needed [1]

The new search system with the updated authentication system is online. There is still some missing functionality but it should let everyone download samples. If you find any problems please let me know. There will be some quirks as we move to the new version of the website. If you find any bugs please let me know on Twitter @openmalware.

Danny

[1] You still need a Google account to download the samples

State of Offensive Computing

I would like to take this time to thank everyone that expressed their support while Offensive Computing was offline. It was a trying time and I really appreciate everyone's support. Without getting into any of the specifics of why the site was offline for two months, we are back and here to stay. There are a couple of people who were instrumental in helping to keep everything up and running. Paul Royal, from the Georgia Tech Information Security Center helped out significantly with hardware and the new home of the site. Kelcey Tietjen also stepped in and helped out tremendously. If you see either of them at some upcoming conferences (hint: Paul is giving a talk at Blackhat) buy them a drink.

There are a couple of changes that are going to happen that more accurately reflect the intentions of the site. First, the name will be changing to Open Malware. The new name more accurately reflects the purpose and intention of the site. Way back in 2005 the intention was to make this a place where you could find information related to malware and other types of hacking. As things (and life) have progressed it has changed into a malware research site, specifically with the ability to download malware samples. The domain will be OpenMalware.org in the very near future.

The second big item of news is that we will be transitioning to a download-only malware repository in the coming weeks. The blog site will be officially shutting down. There are much better forums maintained by commercial services that have taken up the role of a discussion area. Specifically the /r/ReverseEngineering and /r/Malware sub-Reddits, and OpenRCE are better avenues of communication. I will maintain a static version of the site to archive the old content.

To accommodate the new download site, there will be a couple of changes. First, a lot of the back end software has changed. Searches will be faster, more malware will be available, and the overall maintenance will be a lot easier. Second, you will need to have a valid, verified Google Account. Having a Google account allows us to use industry standard authentication, and most importantly not to have to maintain a user database. Get one here if you haven't already. In the meantime new account creation is disabled while we make the transition. Old accounts should work as normal.

Finally, we are discontinuing our commercial services. I would like to thank all of our customers for their business. You all helped to support this site and maintain an open service. We will be looking at transitioning to a non-profit status in the coming years.

Thanks again,

Danny Quist

VizSec 2012 Call for Papers Out

VizSec 2012 will be held in mid-October as part of VisWeek in Seattle. Papers are due July 1.

The International Symposium on Visualization for Cyber Security (VizSec) is a forum that brings together researchers and practitioners from academia, government, and industry to address the needs of the cyber security community through new and insightful visualization techniques. Co-located this year with VisWeek, the 9th VizSec will provide new opportunities for the usability and visualization communities to collaborate and share insights on a broad range of security-related topics. Accepted papers will appear in the ACM Digital Library as part of the ACM International Conference Proceedings Series.

Important research problems often lie at the intersection of disparate domains. Our focus is to explore effective, scalable visual interfaces for security domains, where visualization may provide a distinct benefit, including computer forensics, reverse engineering, insider threat detection, cryptography, privacy, preventing 'user assisted' attacks, compliance management, wireless security, secure coding, and penetration testing in addition to traditional network security. Human time and attention are precious resources. We are particularly interested in visualization and interaction techniques that effectively capture human analyst insights so that further processing may be handled by machines, freeing the analyst for other tasks. For example, a malware analyst might use a visualization system to analyze a new piece of malicious software and then facilitate generating a
signature for future machine processing. When appropriate, research that incorporates multiple data sources, such as network packet captures, firewall rule sets and logs, DNS logs, web server logs, and/or intrusion detection system logs, is particularly desirable.

More information is on the web site:

http://www.ornl.gov/sci/vizsec

Practical Malware Analysis - A Book Review and Curmudgeonly Rant on the State of Reverse Engineering

Recently I was asked to review a pre-publication copy of Mike Sikorski and Andrew Honig’s book “Practical Malware Analysis” by Nostarch Press. I gave it an enthusiastic review, and I strongly believe this will become the defacto text for learning malware analysis in the future. This is a review of that book, and a short rant on reverse engineering.

Before getting into Practical Malware Analysis, I hope you will indulge me in a rant about other books on the reverse engineering topic: They are not pretty. If you’ve taken one of my classes I recommend a few books for learning reversing, but climbing the steep mountain of pre-requisite material before you can attempt to be somewhat proficient is daunting. Specifically the books I recommended were based off of each individual author’s own personal style of reverse engineering with the tools that were available at the time. The field has gotten much more accessible thanks to the awesome tools that are out there from companies like Hex-Rays and Zynamics.

Practical Malware Analysis does a good job of tying together the methods of modern malware analysis. While most of the previous texts have done a good job of presenting the state of the art at their time, PMA overviews many of the tools that are in use in the modern day. Part 1 starts off with the basic static techniques, how to set up a virtual environment, and dynamic analysis. These initial steps are the basis for any good reversing environment. What is nice is that these topics aren’t dwelled on for an entire book.

Part 2 goes over the relationships of the Intel architecture, IDA Pro, modern compilers, and the Windows operating system to reverse engineering. Having an understanding of this as it applies to the reversing process is extremely important. Outside implementing a compiler, learning the fundamentals of the architecture is the most important skill a reverser can have for understanding the field. The difference between an adequate reverser and a great reverser lies in the understanding of how the system interactions work.

The rest of the book is focused on the advanced topics of dynamic analysis. Part 5 deals with all the ways that malware authors can make your life miserable, from anti-disassembly to packers. Part 6, “Special Topics,” talks about shellcode analysis, C++ specifics, and the ever-looming threat of 64-bit malware. I suspect that there will be a second edition once 64-bit malware comes in vogue.

Overall the book is excellent for those that are new to this field. Experts love to curmudgeonly talk about how nothing is new anymore, everything sucks, and pine for the good old days of reverse engineering with some wire-wrap, a lead pencil, a 9-volt Duracell, and a single LED. If you consider yourself one of these people, reading this book is going to feel a lot like wearing someone else’s underwear. If, on the other hand, you read it and put aside your natural skepticism of all things new, you might learn something.

I really do like this book.

Edit 3/4/2012: I have no financial interest in the book. The only thing I received was a reviewers copy. This was not sponsored or paid for in any way by the authors or publishers.
Edit 2/13/2013: There has been a translation to Serbo-Croation of this review by Joanna Milutinovich

Introduction to IDA Python

The Introduction to IDA Python document by Ero Carrera is one of the better documents on scripting the IDA Pro platform available. After talking with Ero directly, I have received permission to host the PDF directly on Offensive Computing to make it available long-term. Enjoy.

Introduction to IDA Python by Ero Carrera

Danny

Three Million Samples

Today we added our three millionth sample to the Offensive Computing malware corpus. While three million pales in comparison to the total malware out there, we still have the largest openly available collection available on the open Internet.

The story of this site has had its ups and downs, and on multiple occasions it was on the brink of shutting down. Every time I heard from someone at a conference, or saw mention of the site in presentations in papers, this helped to keep us up and running. The resources needed to keep things moving have been interesting to deal with. Our commercial services have supported the ongoing maintenance of running a free malware archive.

Some changes are coming to the site Real Soon Now (TM) and I think now is a good time to share them with you. First, the storage and catalog software we have been running on has been sluggish for a long time. I'm about 80% through a rewrite of the underlying malware processing system that should get us to the next order of magnitude without problems. We have made some key partnerships with other open malware resources and we are beginning to put those into service soon. Second, our Reverse Engineering training is getting a massive rewrite. Currently we only do on-site offerings, but we are investigating the possibility of hosting at a more public general venue. Finally, the blog that you see here will be undergoing some changes.

Thank you to all of our customers, users, and supporters. Without you Offensive Computing would not be up and running today. Watch for more news coming soon.

Danny Quist
Founder, Offensive Computing, LLC.

ShmooCon 2011: Visual Malware Reversing

This past weekend I had the pleasure of presenting at ShmooCon 2011. This conference continues to be one of my favorites. Shmoocon is a small conference that is trying very hard to stay that way. This year I talked about my improvements to VERA over the past 6 months. Much of the talk was centered around live demos, which unfortunately did not make it to the slides. The new tracing module and updated versions of the VERA code will be posted here soon.

Video of the Talk
PDF of the Powerpoint Slides
Download the new VERA code here

Abstract:

Reverse engineering is a complicated process that has a lot of room for improvement. This talk will showcase some improvements to our visualization framework, VERA. New features that decrease the overall time to reverse a program will be shown. New items are a debugger based interface which allows for faster analysis without the need for a hypervisor, integrated trace processing tools, IDA Pro integration, and an API to interface with the display. During the talk I will reverse engineer malware samples, and show how to integrate it into your reversing process.

Danny

VERA 0.3 Released

VERA 0.3 has been released. This new version contains a bunch of new features and API improvements. The two biggest updates are the addition of the trace file parsing and analysis inside of the GUI. This alleviates the need for the gengraph.exe program. The next big feature is the integration with IDA Pro. Currently it only supports version 5.6 and 6.0 versions of IDA. Finally, VERA now includes documentation.

Download VERA 0.3 MSI File
Download VERA Documentation (PDF)

Please feel free to email me (dquist at this domain) if you have any comments. Those of you that have responded thank you very much.

Changelog:

* Added processing of trace files without having to use gengraph via new wizard
* Better handling of low memory situations
* Major code cleanup, refactoring, and new buzzwordy sounding tasks
* Added a toolbar, because everyone loves those
* Added IDA integration and IDA Pro module
* Fixed a bug involving parsing of non-traditional Ether trace files
* Now should support larger and more complicated graphs
* I'm getting paid to write and support VERA. :)

At Shmoocon 2011 I'll be rolling out the next version of VERA, complete with new features.

Vera 0.20 - Now Available

After a lot of work, I'm happy to announce that Vera 0.20 is available for download. This release is a rewrite of the entire code base into wxWidgets. Based on some excellent feedback from my talk at REcon (an excellent con by the way) I've made some substantial changes to the backend code.

If you're not familiar with VERA, it's a visualization tool to help understand the dynamic execution of a program. It's made to take the instruction traces from Ether and generate directed graphs showing the overall flow and composition of a program. Identifying the OEP is easy, as well as looking for main loops and initialization sections of the program. You can read about VERA in my Vizsec 2009 paper for more information.

Here's the complete changelog:

Rewrite of entire codebase to wxWidgets (should allow for future ports to other platforms)
Added configuration file (~/.wxVera/wxvera.ini)
Read/save previous window position and size from/to config file
Fixed a graph centering problem
Added update checking code
Reloading of graphs more efficient
Added welcome message
Introduced notebook style for GUI

Please feel free to contact me (dquist at this domain) if you have any problems or suggestions for VERA. Thanks!

AV Testing Standards: Don't Like the Results of the Tests? Change the Rules

There were good responses, mostly from people in the AV industry, to my blog post about the malware testing standards. Overlooking my error linking to their original paper (sorry) there were some points I would like to address.

At the heart of this whole process, is exactly how dangerous a collection of malware is. For the consumers, I would argue, it's not dangerous at all. The malware industry is the only one who has to fear from it. Notice I didn't say just the AV vendors, but also the producers of the malicious software. In large part the authors depend on a closed, inside group of people unwilling to collaborate openly on the problem. If you look at the major sources of malware in academic research prior to the creation of large open collections, you'll see that there were some big problems. First, the samples were old and not representative of current threats. Second, those samples either did not work or were not malicious in nature. Finally, the samples are traded as something of value.

I'm no different, of course. I derive value both from the collection and from consulting. I do, however, go out of my way to support those doing open research as much as I can. If someone in academia needs access to samples, just contact me and I'll work something out. Likewise we have helped innumerable small businesses get their start in the malware world before they could enter the "circle of trust" mentioned by David Harley.

The "circle of trust" is often cited when discussing who can and cannot gain access to these samples. Over the course of the years I've joined four of these groups. While the vetting is done as best as possible, there's very little outside of an email address, and a recommendation keeping someone from joining. Antivirus vendors exchange malware with themselves at a much higher volume, but there is still a perceived difficulty of entering this area. Malware exists on the Internet in a freely available manner as a function of its being. Limiting sample access to a certain set of privileged people fundamentally hurts innovation and response by everyone.

There was also some allusion that I did not support malware testing at all. That is not the case. Malware defense systems should be heavily tested against a range of threats. The basis for my problems with the AMTSO is that it should *not* be composed of anyone in the AV industry. Consumer Reports did an excellent job exposing the ineffectiveness of AV vendors by producing new samples. Due to the very nature of the threat, there are going to be new samples that are discovered for the first time. If an AV software can't respond to this threat, it should not be given a favorable review.

The current set of players in the malware testing arena are profit driven. In and of itself that's ok, I'm all for capitalism, but in fairness there needs to be an independent authority. AV testing companies that publish open information on the effectiveness of scanning results are not independent. Without naming names, there is a prominent one claiming to provide results for the public, but instead is backed by every AV vendor in the industry. This testing company takes in new samples, scans them with all the products, then tells the vendors how their performance rates. What is not acceptable, in my view, are the shoddily written reports intended for consumers that report unethically high detection rates.

Finally I would like to address the ethics of the malware tester. One thing I agree with David Harley on is the need to represent the full scope of the testing process to the consumer. One of the things that the academic world does well is to produce research which can be recreated by other researchers. That's the intent, at least. AV testing standards advocated by the vendors cannot and will not provide the latest samples to malware authors. What this ends up doing is providing all the methods of testing, but not the actual data to test on. For those of us able to use new samples, it's not a problem. Others who have older data and are unable to acquire new malware (due to cost, time involved, etc.) are left with only one viable option: Synthesize new samples using the exact same methods available to the authors.

Syndicate content