NSRL Banner

NSRL Logo

HOME

GENERAL INFORMATION

TECHNICAL INFORMATION

DOWNLOADS

CFTT Website

Privacy Policy/Security Notice
Disclaimer | FOIA

NIST is an agency of the
U.S. Commerce Department

Date created: 8/20/2003
Last updated: November 3, 2010

Technical comments: nsrl@nist.gov

Website comments: web897@nist.gov

 

 

 

NSRL Frequently Asked Questions

Welcome to the NSRL FAQ. Below you will find answers to some commonly asked questions regarding the NSRL. If you cannot find your question, please feel free to contact us.


The tool I'm using gives me an error when I use the NSRL hash set
Can I download the NSRL hash set?
I can't download the hash set. Would you mail me a CD set?
What is the format of the NSRL hash set?
Is there a listing of contents in the NSRL hash set?
Can I download the NSRL voting software hash set?
Where do you get the software in the NSRL and how can I get my software included?
Can I borrow software from the NSRL?
I'm Federal Law Enforcement, and REALLY need a copy of something you have...
I have a question regarding my subscription (payment, receipt, etc.)
Does the NSRL contain hashes of illicit images (e.g. child pornography)?
Where can I get hashes of illicit images or malicious items?
What's the difference between the NSRL and Hashkeeper?
Do you plan to migrate to or include other types of hash (e.g. SHA-256, Whirlpool) in the future?
Which hashes are for known bad files?
I found (6, 10, 12) files that give me KFF alerts, they claim to be hacker files. Is this machine hacked or used for hacking?
Which hashes are for (hacking / accounting / etc.) programs?
Does the RDS include installed software results/installed file hashes?
Who uses the NSRL hash set?
Can we have some of the files in your library to test an algorithm?
A High-level Illustration of the NSRL


The tool I'm using gives me an error when I use the NSRL hash set

Before shipping, we make each release available to vendors of leading forensic tools for testing against their products. For completeness, we also test the data in house against some of the major forensic tools. Our testing is done on contemporary desktop systems running Windows XP.

We try to resolve some questions, but in our experience the vendor support staff will likely give you a better response.


Can I download the NSRL hash set?

Each release of the NSRL hash set is made available for download free of charge one month after the subscriber CD release. The download comprises 4 ISO files which you can get from our downloads page. The ISO files can be downloaded and burned as a 4-CD set. You should verify the downloads are correct by comparing the value in the iso_hash.txt file to that published on the downloads page. You should also verify that the hash set is correct by using the hashes.txt and version.txt files you find on the CDs.


I can't download the hash set. Would you mail me a CD set?

We would be glad to send you one hard copy via postal mail. Please provide a mailing address.

After this, you will need to download or subscribe to acquire future releases.


What is the format of the NSRL hash set?

The format of the data is described in our paper Data Formats of the NSRL Reference Data Set (RDS) Distribution (PDF).


Is there a listing of contents in the NSRL hash set?

The full content lists are updated with every quarterly release. You can find links to the Product listing, Manufacturer listing, and Operating Systems listing on the downloads page.


Can I download the NSRL Voting Software hash set?

Yes. The voting RDS files are located here. There is also a separate NSRL voting FAQ.

You should verify the downloads are correct by using the hashes.txt and version.txt files.


Where do you get the software in the NSRL and how can I get my software included?

We purchase most of the software in the NSRL. We try to get everything on major retailers’ top selling lists. Some software we hear about by word of mouth, some by schedule (like tax programs each tax year, security, antivirus) and some by requests from law enforcement and other agencies. We accept donations from manufacturers and have paperwork to state we will not use the software license (donors are recognized on our website). All donations of new software should be COTS shrink-wrapped and exactly what a consumer would purchase. We accept donations of used software as long as it is in useable condition but there is no guarantee that it will make it into the NSRL RDS. We do keep a limited number of duplicate software for media degradation testing and in order to keep a back up of the most popular software, such as operating system packages.

To donate software to the NSRL, please mail packages to:

NIST
NSRL Project
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970



Can I borrow software from the NSRL?

We apologize, but we cannot lend out copies of the software in our collection.

We make the hashes (MD5, SHA, etc.) available to everyone, but the software itself is (a) stored in an evidence locker, (b) is often donated by vendors with a non-use agreement, or (c) can't be redistributed due to copyright.

However, our experience suggests you might want to try hitting tech swap meets, used bookstores, bargain bins in non-chain stores - they've been a gold mine for us.


I'm Federal Law Enforcement, and REALLY need a copy of something you have...

We will do what we can within the bounds of the licensing of contents of our collection.


I have a question regarding my subscription

Please contact our subscription department.


Does the NSRL RDS contain hashes of illicit images (e.g. child pornography)?

No. The NSRL is prevented by law from handling such files, and NSRL policy prevents us from including the hash of a file in the NSRL RDS unless we possess the original copy of that file.


Where can I get hashes of illicit images or malicious items?

The NDIC HashKeeper project is one source of illicit data hashes (see below).

NIST also has a catalog of digital forensics databases which you may find useful.


What's the difference between the NSRL and Hashkeeper?

The NSRL RDS and the NDIC's Hashkeeper are collections of File Identification Information (FII) which are typically used to identify computer files during forensic investigations of computer systems. The principal differences between the two collections are as follows:

  1. Provenance. All NSRL data is derived from purchased or donated software which is retained in a secure facility at NIST. NSRL FII is thus traceable to the original software. Hashkeeper is a central repository of FII donated by various sources, usually obtained by law enforcement during the course of forensic investigations of suspect systems. Hashkeeper FII is not traceable to its source.
  2. Court worthiness. The FII published by the NSRL is designed to be admissible as evidence in a U.S. court of law: NSRL data is traceable and verifiable back to the original software packages which are maintained under evidence locker conditions. The Hashkeeper collection is designed to be of use to forensic investigators but is not generally admissible as evidence in a court of law.
  3. Scope. NSRL data is strictly limited to that which can be traced to physical installation media which can be obtained and held as evidence at NIST. Hashkeeper data has a potentially broader scope, dependent only on the nature and quantity of FII that law enforcement officials decide to donate.
  4. Illicit file data. The NSRL is prohibited by law from obtaining and storing certain forms of illicit file data (e.g. child pornography images) and contains no FII on such data. The Hashkeeper collection is known to contain FII on illicit file data.
  5. Specificity of data. The NSRL uses five measures to identify each file (the SHA-1 cryptographic hash, the MD5 cryptographic hash, the CRC32 checksum, the file name and the file size), whereas Hashkeeper supplies only the file name and the MD5 cryptographic hash. There is thus a lower level of confidence in the Hashkeeper data.

Do you plan to migrate to or include other types of hash (e.g. SHA-256, Whirlpool) in the future?

Yes, we will be collecting SHA-256, Whirlpool, and several other pieces of metadata that we don't gather now.

The additional metadata will be included in a separate product - the RDS will continue in its present format.


Which hashes are for known bad files?

The members of our steering committee (federal, state and local law enforcement) consider the files in the NSRL database as "known" - NOT "known good" OR "known bad" - just "known application files."

NIST does not make a decision about "known bad' or "malicious" or "notable", because there are various case scenarios where that classification is not cut-and-dried.

Note, however, that the NSRL database does contain hashes of files from applications which are traditionally viewed as malicious (encryption, steganography, hacker tools).

You can partition the applications according to your specific needs using the "ApplicationType" field in the "NSRLProd.txt" file - if you consider steganography apps as bad, you can identify them as such using that data.


I found (6, 10, 12) files that give me KFF alerts, they claim to be hacker files. Is this machine hacked or used for hacking?

We have had reports from several investigators that a small number of files - on the order of 10 or 12 - will cause "alerts."

It is our opinion that someone unknown to us has designated all of the file hashes associated with some NSRL hacker applications as "notable" or "malicious" (probably inside a tool that imports the NSRL hash set). Unfortunately, a few of the files used by those hacker apps are very common files used by normally harmless software. If you have a small number of "alert" hits, it is very likely that those are false positives.


Does the RDS include installed software results/installed file hashes?

No, installed software results are not included at this time.

Collecting installed file hashes is a very labor-intensive process. We hope to have a somewhat automated process to aid in collecting these in the future, but we do not collect them in any bulk manner right now.

The NDIC Hashkeeper collection does have installed file hashes - see above.


Which hashes are for (hacking / accounting / etc.) programs?

You can look in a file called NSRLProd.txt and find a column called "ApplicationType". We have classified the programs, and you can look for the description of your interest - steganography, keylogger, office suite, etc.


Who uses the NSRL hash set?


Can we have some of the files in your library to test an algorithm?

No. Hovever,
the NSRL has a research computing environment containing some 18,000,000 unique original files, along with a database containing metadata about the files (filename, bytesize, etc.).

The format for running an algorithm against the file collection is basically that you would submit a job - in the form of your code - to the NSRL. We would then run your job against the file collection, returning the results and your code to you upon completion.

There are various conditions of access to the research environment, including:

  • NSRL staff will review the code prior to agreeing to run it
  • the research environment is not connected to any external network
  • the original files cannot be modified
  • no part of the original files can be copied, e.g. exported into the result set
  • the result set generated by your job may be made public

Please contact us for details.


A High-level Illustration of the NSRL

A High-level Illustration of the NSRL