The better way to handle this section of code is:

  while (treeWalk.next()) {
    switch (treeWalk.getRawMode(0) & FileMode.TYPE_MASK) {
    case TYPE_MISSING:
    case TYPE_TREE:
    case TYPE_GITLINK:
      continue;
    }

That also lets you avoid the isSubtree test above.