This causes us to always dive into all ignored folders.

If the ignored folder is a build output directory, we may be looking at millions of file paths that are all ignored, and that we don't care about, anywhere.

By putting the check after the DirCache code below we can safely skip ignored directories, but only if there are no tracked files within the ignored directory.  (If a file is tracked, it is should always be looked at, even if it is also ignored.)