Digging in Xen to fix some problems

I had some trouble restarting a xen user domain tonight. The shutdown, destroy, console, and list xm commands were all simply hanging. I finally tried restarting the xend service which fixed the hanging problems. But when I tried to start the user domain that originally gave me problems I got this error message:

Error: Device 769 (vbd) could not be connected.
Device /dev/vg/lv is mounted in a guest domain,
and so cannot be mounted now.

To fix the problem I had to use the xenstore tools which I found to be poorly documented. The trick was first located the backend parameters for the disk. I was able to find this information with the following command:

• xenstore-ls | pager

I then searched for the disk name until I saw something like this:

backend = “”
vbd = “”
10 = “”
2049 = “”
params = “vg/lv”

I was then able to get the user domain to start again by running:

• xenstore-rm -t backend/vbd/10/2049

Not very much fun and I certainly hope this isn’t a common event. I’d like to think Xen is more resilient than this.