After recently upgrading from 6.0.1 to 6.1, we are unable to build machines that call the stub workflow WFStubBuildingMachine. This worked properly prior to the upgrade. If we remove this workflow, then machines build okay, but since we have numerous items occurring in vCO (vCenter annotations, AD group created, new machine name assigned, etc.) that are called by that stubworkflow, machines build without it are useless to use.
Here are the 3 stub workflows that we have configured via custom properties. Note: We do not have any values assigned, but never did before the upgrade either. I assume a value is not required as long as you are okay with vCAC picking the latest rev of the workflow when it runs?
After requesting a machine be built that includes calling WFStubBuildingMachine, it eventually times out and the following error is shown in the logs:
We've tried the following already without success:
- We have a ticket open with support. A sev 1 for two weeks now. The ticket is with engineering. Substantial amounts of logs have been provided to support.
- We've removed that stubworkflow from blueprints by removing the custom property and machines build fine.
- We've left other stubworkflows in the blueprints and machines build fine as long as WFStubBuildingMachine is not present.
- We are also having an issue with Designer after the 6.1 upgrade. I have a separate post regarding that as we're not sure if it's related or not. It seems as though it may be unrelated as other stubworkflows work fine, just not this one. vCAC Designer (cloudutil.exe) will not launch after upgrading to 6.1
- We can't use Designer to crack open the WFStubBuildingMachine workflow since Designer is broken (see #4 above).
- Everything else post upgrading to 6.1 seems to be working fine. All tabs in the UI are functionality. All services that you'd expect to be running are running on the appliances and IaaS server. Machine decomissioning for existing machines or machines built without WFStubBuildingMachine work properly, etc.
- vCAC 6.1 vCO plugin has been installed/upgraded. This issue existed before we did the vCO plugin update and still exists afterwards.
- vCAC Workflow Package fix has been applied. This issue existed before we did the vCO plugin update and still exists afterwards.http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2088838
- When upgrading to 6.1 we selected the option to 'preserve my latest workflow versions'. We can see at the CLI via cloudutil.exe that it added a new rev for the 6.1 WfStubBuildingmachine workflow (rev 8), then copied out rev again as the latest (rev 9). Our previous rev was 8 prior to the upgrade. So it seems like this worked as designed.
We are down hard because of this. We'd prefer not to roll back even though we have good backups, as if no one else is experiencing this issue then VMware will likely not get to the root of it and we won't be able to upgrade to 6.1 down the road with confidence.
One thing we are curious about is whether the vCO customization workflow as shown in this post below hasto be run again after upgrading from 6.01 to 6.1? We didn't think so as it's not referenced in any upgrade guides, documentation, or blog posts, but just curious if anyone else did it. We didn't want to do it as it will wipe out what changes have been made to the stubworkflows prior to it. http://www.vmtocloud.com/how-to-extend-vcac-with-vco-part-1-installation/
Has anyone seen this or anything similar? Even if you haven't, do you have any suggestions to try?