Alternatively, I can just use the SEEN flag like everything else.  I was pretty sure the depth flagging covers it correctly, but I had to think about it for a long time as well, and I'm still not positive I'm right, so all in all that's probably a sign that it should just be done a simpler way.