So I was working on a script to poll the VRO API for workflows, which would essentially track these until completion, and archive them to a database. The logic is basically:
1. Look at vRO and get a listing of executions for a given workflow, and get the details (execution Id, parameters, completion time, completion status, last step, etc.)
2. Check the database table where I am storing these to see if this execution Id has been logged
3a. If the execution Id is already logged, update the entry.
3b. If the execution Id is not already logged, insert the new entry.
4. Check the status of the execution - if it is in a completed state of some form (failed, completed), delete that entry from VRO..
The script worked great for logging - and it worked perfectly fine right until the point that I started deleting workflows.
I am basically running:
DELETE https://{vro_server}:8281/vco/api/workflows/{workflow_id}/executions/{execution_id}/
I am running this using Invoke-RestMethod in Powershell. I also noticed that in the relations.link property of the return JSON when retrieving a execution, there are some other verbs available:
https://{vro_server}:8281/vco/api/workflows/{workflow_id}/executions/{execution_id}/logs/
https://{vro_server}:8281/vco/api/workflows/{workflow_id}/executions/{execution_id}/remove/
https://{vro_server}:8281/vco/api/workflows/{workflow_id}/executions/{execution_id}/state/
I had tried playing with the /remove/ resource, but I tried GET, PUT, PUSH, and DELETE, and it threw a 404 on each. So it's entirely possible I'm going about this the wrong way, and if so, feel free to stop me here.
So, first I tested the DELETE verb against the execution. This worked on my test execution. So I added the logic into the script, added a long sleep, turned on verbose output, and kicked it off and watched. There was 2 successful deletes right at the beginning. It moved to the third, and I noticed it took longer. Eventually I got an error indicating that the operation timed out. It failed on the 4th as well. I killed the script, and after making a couple modifications, I kicked it off again. This time, even getting my listing of executions failed with a time-out issue. It then moved on to my next VRO appliance (I have 4, because they only support 5 vcenter SDK connections each), it successfully pulled in all my executions, and then just like the first appliance, there was 1 success, then failure. Then, a second failure. Then a success. Another failure, another success. And then it moved to all failures. Now if I try hitting the REST API for either of these appliances, it fails.
What gives? Did I do something wrong? How do I start this sucker back up?