More Benchmarking

Used perl -d:DProf and dprofpp to gather stats on runs
with small files and large files (100MB to 2GB)

Most time taken in dealing with system calls to the hashing .EXE

Added functionality to deal with archive files based on Magic file info

Overall speed went into the toilet using File::MMagic
dprofpp using magic

One MS MSDN DVD took 30 hours to hash without magic

Same DVD took a week using magic - over 650,000 files

Still using Perl 5.6 at this point, heading to 5.8 to resolve Unicode issues

April 2003 attended a talk by M.J.Dominus

• Perl 5.7.2 has Time::HiRes
• strace -s 10000 -f -o foo.trace perl foo > /dev/null
  grep execve foo.trace
• Devel::Small Prof - less acurate
• arrays are faster than hashes
• each consumes space not time with respect to keys

Bottom line: when it's slow, we know why and can offer the user options to trade time for completeness

We always choose completeness - NSRL motto: "You can never have too much overkill."