Friday, December 18, 2009

Problems with RHEL Xen and rhn update

I'm running a Dell 2950 with RHEL 5.3 Xen as my virtualization host, with a half-dozen guests.  There's a plan in place to move them all to kvm - but that's another post.

Yesterday, I got this ominous message when doing a redhat-update:

Error Message:
    Abuse of Service detected for server defiant.denmantire.com
Error Class Code: 49
Error Class Info:
     You are getting this error because RHN has detected an abuse of
     service from this system and account. This error is triggered when
     your system makes too many connections to Red Hat Network. This
     error can not be triggered under a normal use of the Red Hat Network
     service as configured by default on Red Hat Linux.

     The Red Hat Network services for this system will remain disabled
     until you will reduce the RHN network traffic from your system to
     acceptable limits.

 Abuse of Service?? Oh, no!  What have I done!

Well... nothing.  Red Hat did.

There's a cron job installed in /etc/cron.d when you're a Xen host, called rhn-virtualization.cron.  It monitors the hypervisor and notifies Red Hat Network if there's been a change in status in any of the virtual guests, so that RHN can make sure your systems are up to date.  Sounds logical.

In a recent update, this was changed.  It now reports back to the mothership any time any of your guests flips from blocked to running, and vice versa.  In other words, lots.  Really lots - any time a guest is waiting for keyboard input, or disk IO, or CPU, or...

So if your machine checks in more than 100 times per day after the first 1500 checkins, you get flagged as abusive.  And with a half-dozen machines running full out, I'm certain that my machines are checking in more often than that.  Boom, error 49, and it's a real pain to get your machines re-registered.

Suggested fix is to change the cron job to once an hour or so.

My suggested fix is to nuke the cron job, and run

/usr/bin/yum update --security -y

as a daily cron job.

No comments:

Post a Comment