NSRL ssdeep Datasets

HOME

GENERAL INFORMATION

TECHNICAL INFORMATION

Technical Papers
Presentations

DOWNLOADS

CFTT Website

NIST is an agency of the
U.S. Commerce Department

Date created: 1/20/2010
Last updated: 1/20/2010

Technical comments: nsrl@nist.gov

Website comments: web897@nist.gov

ssdeep Datasets

On this page we will make links available to datasets that have applied the ssdeep algorithm (aka "fuzzy hash") to NSRL files.

Data sets using other algorithms are aavailable.

A text file which relates SHA-1 hashes to ssdeep 2.0 signatures is available.
A 650 MB Zip file can be downloaded which contains a 1 GB text file with 8,334,077 rows.
Each row has a SHA-1 hash, a tab character, and an ssdeep CSV output.
The SHA-1 values can be matched to SHA-1 values in RDS 2.27.

Example rows:

00000142988AFA836117B1B572FAE4713F200567        768:Cv9oR/2r/2LMZbZosLapUbFx7ECQsWMFvDiu:Cv93r/2LMZ9oLpqjdQsWMFvWu,"008221197"
000001EF1880189B7DE7C15E971105EB6707DE83        24:q2FpFDSECVH4elBEThBhID+r7Pyzj3RFwim4dnBWopLr5urJVlHYSLR5AfHM:qqnDSECl4ejgeDQ7Oj3LjnB5paYfHM,"011942112"
0000051C1E5A5DECF28FE8B57ABFE2B82A3EDD1C        96:0sdrckFsyVm/q4DDDDDDDDXrgytWeU4DDDDDDDD0ZlCpIzsUl9piceLoIuD5n45:DrXFJVaRIz3NZpe,"008466494"

File signatures:
SHA1(SHA1-ssdeep-RDS227.zip)= ef1542c272a5394cb03d451bbadcb11c152edd4d
MD5(SHA1-ssdeep-RDS227.zip)= 16e0efe671643f21be9be79d3f870335

SHA1(SHA1-ssdeep-RDS227.txt)= b9b94cfb497b0922a94e8d0332bd98b22cdf4f4e
MD5(SHA1-ssdeep-RDS227.txt)= c24752a0885f7e795847fc97b7380ad6

NSRL has hashes of blocks of files.

A text file is available which relates the SHA-1 of a complete file to the ssdeep of the first 4096 bytes in the file (provided the file is that large).
A 400 MB Zip file can be downloaded which contains a 720 MB text file with 5,506,973 rows.
Each row has a SHA-1 hash, a tab character, and an ssdeep CSV output with a prefix.

Example rows:

00000142988AFA836117B1B572FAE4713F200567        ssdeep[4096]=96:KKXFKCDU0kEnsSMlBSjfLUrgT9CvBXnJIZcgcFSqou:ZKCiBxgjfkvBZIZcLFSqou,"83765"
0000051C1E5A5DECF28FE8B57ABFE2B82A3EDD1C        ssdeep[4096]=96:0sdrckFsyVm/q4DDDDDDDDXrgytWeU4DDDDDDDD0ZlCpW:DrXFJVaRW,"60818"
000009DFA0C5DD9FCBFD4BDE20902A90B11E8228        ssdeep[4096]=48:vOqYV1Nznv38RDmzPsxGruOdAW2+aMQEoW:vOqYV1NDvMRD+sxGruO2W24QTW,"9389"

File signatures :
SHA1(ssdb4096.zip)= ad6226c887fe84b5991c4a6030ca67065b59bcc3
MD5(ssdb4096.zip)= 3003554697a68c2f56b2cc7b8ed4716a

SHA1(ssdb4096.txt)= 565406df6f24a9ff75e2e631b648fdf6e3973b4b
MD5(ssdb4096.txt)= c1cdb8fd272e42480f3a70f76020ab13