11-27-2013 12:16 PM
I have a customer who is having sync problems, they are on v8 latest patches and use http sync.
They recently ran out of disk space, I know this should not happen but it did.
When running sync it almost immediately displays a synchronization failure, out of memory.
They had 400K WGLOGS and I was receiving the above error when running sync so could not process them, in addition outfiles had 40k files in it, so I attempted to transfer the outfile to the http outfiles ONLY and forget processing the WGLOGS, however to compound the issue, when running the transfer up I keep receiving the following error.
Failed to send file. The handle is in the wrong state for the requested operation
Sync stops processing, the tef files are ok and re-running sync the file is sent but it randomly stops on another file with the same error, I had on a previous occasion seen this same error and it only appeared to occur when the outfiles had a lot of files, I researched the error but could not find anything on it, I also raised with Sage support (at the time) and they could not offer any insights on the particular error, I eventually got the outfiles empty and all then worked as expected with no errors, I did find the customer had not excluded AV from the sync folders and put it down to this, however AV is excluded.
I am not seeing anything in the event logs, and nothing out of the ordinary in the IIS or httperr logs, memory is fine, as is disk space.
So just to recap currently there are a large amount of que files being processed by the slxserver, trying to process the WGLOGS gives me an out of memory error, I wonder if this is due to the logging server constantly transferring the files to WGLOGS?, When trying to transfer the files up, it gives me the Failed to send file. The handle is in the wrong state for the requested operation.
As things stand I can't leave sync running to process the files as it is failing with one or other of the errors (depending on what I try to process) So I am having to baby-sit it but am concerned at this rate, it is never going to catch up.
Has anyone encountered either of these two errors or have any ideas on what could be causing these?
Any and all suggestions would be gratefully received and very much appreciated.
11-27-2013 12:32 PM
Running out of disk space is about one of the worst things that can happen w/sync.
At a minimum.. after you recover a BUNCH of GB space back.. it will eventually catch up and (hopefully) not have any errors or loose data.
Worst case.. you have to fix the sync and re-cut every remote.
.. bottom line -there is no easy way out of this one - been there done that MANY times.. and have implemented "free space warning" scripts in synch to prevent this from happening. You can have "things run" on Before Sync as well as After Sync. These can be "blocking" or "non-blocking" setups.. and a free/low-space BEFORE Sync check that Blocks is what you want.
11-27-2013 12:38 PM
Thanks for the quick response, I appreciate running out of space is never a good thing, we do now have plenty of space, however due to the errors, failed to send or out of memory, I can't see at this moment how it is going to catch up, as it is constantly falling over with one or other error.
Do you have any insights on either of those errors?
11-27-2013 12:42 PM
11-27-2013 12:53 PM
Thanks for the idea, I will certainly bear it in mind, currently the outfiles is now down to approx 3k (although I expect this to increase hugely when the que files and WGLOGS eventually get processed) and is running for longer than previously before failing with the failed to send error, so from that perspective at least for now I think I will stick it out for the remaining files.
Have you come across either of those errors before?
11-27-2013 02:56 PM
Thanks for the response, I agree http sync does seem a little slow in general (and certainly at the moment), All sync file locations are excluded, however since this began I disabled AV on access scanning to be sure.
At present the outfiles are empty, however I am still getting an out of memory error as soon as I try to process the WGLOGS, hoping when the que files are all processed that this will allow the WGLOGS to then be processed.
11-27-2013 05:43 PM
"...out of memory error ..."
Seen this caused by a corrupted conflict resolution file.
AFAIK - theonly "fix" was to destroy the file and let the system create a new one (conftran.stm).
11-28-2013 03:22 AM
Thanks for your suggestions, after the que files all processed I was still getting the out of memory error, IF I ok the error and cycle sync again without closing it, it would move on to another file and same issue, went through quite a few using this process and each time received the same error, however if I closed the server and reopened, it would go through all of those same files again.
Checked through the files it was trying to process and found numerous were corrupt, although not all and not all in the order it was processing them, removed them one at a time in the order of processing and after about half a dozen it started to process them (even though in the previous cycling though as above was much more), so it is at present chugging through importing 127K WGLOGS.
Clearly however there is missing data from the corrupt files which ultimately means we will almost certainly have to recut anyway.
I would be very interested to hear if anyone else has encountered the "Failed to send file. The handle is in the wrong state for the requested operation" error previously as at the moment I am not at the stage of sending the files out and from previous experience with this error it seems to occur when there are large amounts of files in the standard outfiles transferring to the http outfiles and I am going to have large amounts to be transferred again, so suspect I am going to hit this issue again.
If anyone has any insights on that error, they would be much appreciated.
Thanks again for all your thoughts and suggestions.
11-28-2013 07:15 AM
What is the exact version/sp/build of yoru synch exe?
.. and agree w/Mike - ftp synch is faster for some odd reason... but "network synch" is something to be avoided at all costs.. big security issues here.. (not a SLX problem.. but a MicroSquish issue).