Visit the NIST Main Home Page

National Software Reference Library Logo

HOME

GENERAL INFORMATION

TECHNICAL INFORMATION

DOWNLOADS

CFTT Website

Privacy Policy/Security Notice
Disclaimer | FOIA

NIST is an agency of the
U.S. Commerce Department

Date created: 1/20/2010
Last updated: 1/25/2010

Technical comments: nsrl@nist.gov

Website comments: web897@nist.gov

 

 

 

Non-RDS Algorithms

On this page, we will make links available to data sets that use hashing
or digest algorithms not contained in the RDS release.

If there is an algorithm or process that you think would be interesting to run on the 16,000,000 files
found in the RDS (or on a subset of files), please contact nsrl@nist.gov .


Data sets using the ssdeep algorithm (aka "fuzzy hashes") are found here.


A text file which relates SHA-1 to SHA-256 hashes is available.
A 760 MB Zip file can be downloaded which contains a 1.3 GB text file with 11,068,076 rows.
Each row has a SHA-1 hash, a tab character, a SHA-256 hash, a tab and a filename.
The SHA-1 values can be matched to SHA-1 values in RDS 2.27.

Example rows:

00000C818DD9203A38348D260CDB6D7D1A84326A	B3CAFC6AE323C837BFAB5C3B0E9B1C8FC3602581623C7AC4396DE740858608C2	listing53.html
00000E294B89F381C95890A54417CFC6296C15A5	7336B50E69C456EBA58B28912E49E3EFB9025F6393D501796EE1C37B4EDB961C	8c88fe9d-2d33-42af-9d4c-ed00a6e60901.jpg
00000FF9D0ED9A6B53BC6A9364C07074DE1565F3	19E88A5365D6A54E92446A9A12DB16510FF37376FCB25E034C505373B2015EE8	cmnres.pdb.dll

File signatures:
SHA1(rds227-sha256.zip)= ef3a4bc53b39a20e3b70e992aba66fa44566dd96
MD5(rds227-sha256.zip)= be9d97a2a3cd3f381084b9c7ee506b4d

SHA1(rds227-sha256.txt)= 1408015d7560247ef2ed282214502e76b36100cb
MD5(rds227-sha256.txt)= b9efa7a5085aec1dbbc5930b10e26c98

Note: SHA-256 test data is available.


NSRL has hashes of blocks of files.

A text file is available which relates the MD5 of a complete file to the MD5 of the first 4096 bytes in the file (provided the file is that large).
A 177 MB Zip file can be downloaded which contains a 315 MB text file with 5,019,700 rows.
Each row has an MD5 hash, a tab character, and an MD5 hash.

Example rows:

FFFFF06ED6CEB23B016780FD24DFE620        4CF962C2E3815626FE28DE7B2918D8CF
FFFFF19D706C0A754EBB553FEB03C724        D0CDC4D740BDF851D61E049D1DCFA139
FFFFF517FFD3E7425F3F15498595DA89        F5DCF00B42DFE803BD8114798FD67033

File signatures :
SHA1(md5b4096.zip)= 6d99e21fc264cf34edcc0944412f8d0d6eb846e8
MD5(md5b4096.zip)= f12c8e593848e5cb00b6a21c6aed1ab8

SHA1(md5b4096.txt)= eea04645f33434a74872b0ad451e712b5dc07c05
MD5(md5b4096.txt)= f484c2dfee454ddd441fc5cd94202413


NSRL has hashes of blocks of files.

A text file is available which relates the MD5 of a complete file to the MD5 of the first 8192 bytes in the file (provided the file is that large).
A 135 MB Zip file can be downloaded which contains a 240 MB text file with 3,820,376 rows.
Each row has an MD5 hash, a tab character, and an MD5 hash.

Example rows:

FFFFF06ED6CEB23B016780FD24DFE620        41805D5E6AEC565158DEEBB5AD8762B5
FFFFF19D706C0A754EBB553FEB03C724        AA8A7BFF6361797B1A06B05F66059644
FFFFF517FFD3E7425F3F15498595DA89        4B8F27CE545C766DF6E62309EF2A4A9F

File signatures :
SHA1(md5b8192.zip)= f395979ea2476e9e4d45645c95b3a9636a440ffc
MD5(md5b8192.zip)= 08067030ee4f11a8e9b9622a950130b5

SHA1(md5b8192.txt)= 128e124ce3f9c02e7be2786cdd5fb47a557b43c4
MD5(md5b8192.txt)= a79519d481aa73288550c980f9f99c55


NSRL has a SHA-1 - to - filename mapping available (Jan 2011).

A text file is available which relates the SHA-1 of a file to a file name.
A 600 MB Zip file can be downloaded which contains a 1 GB text file with 18,840,521 rows.
Each row has a SHA-1 hash, a space character, and a filename string.

Example rows:

0000004DA6391F7F5D2F7FCCF36CEBDA60C6EA02 00br2026.gif
000000A9E47BD385A0A3685AA12C2DB6FD727A20 femvo523.wav
00000142988AFA836117B1B572FAE4713F200567 J0180794.JPG

File signatures :
SHA1(SHA_name.txt)= adc0c597e97fe224bbbe8b366b5cf6b941fd6aca
MD5(SHA_name.txt)= a95157eb30acfe39cdfaa7729de4eef5