If I got you right you want to keep dialog open in case of validation errors?
Than you can add the following code somewhere in your page
<h:panelGroup id="global_flag_validation_failed_render">
<h:outputText id="global_flag_validation_failed" value="true" rendered="#{facesContext.validationFailed}"/>
</h:panelGroup>
And upon clicking your submit button render the above block like this:
<f:ajax onevent="closeMyDialogIfAllOk" render="@form global_flag_validation_failed_render"></f:ajax>
and in your js code
function closeMyDialogIfAllOk(data){
if(data.status === 'success') {
if($("#global_flag_validation_failed").length === 0){
$("#dialog_id").dialog("close");
}
}
}
For more info take a look over here: How to find indication of a Validation error (required=“true”) while doing ajax command