Sure we can.  You just can't use branch configuration to do it.  Instead the caller must give us options like a remote name or URI, and optionally a remote branch as well.  These override the configuration defaults.