Jump to content
firstimpression

Maintenance and bucket tables question

Recommended Posts

Hi,

Is there any documentation how exactly the "pruning" of bucket tables work?

My issues right now:

1. Sometimes bucket tables have leftovers because maintenance script did not run at a certain hour. An expected behavior would be that any subsequent run would take care of everything that is left unprocessed in the bkt tables. But in reality I have to manually run the "republish.php" script to take care of those leftovers. 

2. Let's say I have 1 master server and 2 slave servers. If the maintenance runs in the order: maintenance-distributed on slave 1, maintenance on master, maintenance-distributed on slave 2 - it looks like the data pulled from slave 2 would be left in master's bucket tables forever (until I run on republish.php). The cause is that after the record in "log_maintenance_statistics" is made, the maintenance would not do anything again for the same hour.

Any ideas how to solve that?

 

 

Share this post


Link to post
Share on other sites

HI @firstimpression,

No, no documentation on this.

1. You should only have leftovers in the event that data arrives in the tables AFTER the maintenance engine has run to summarise stats for that hour. If you fail to run maintenance for a given hour (or two, or however many), then the next run is supposed to figure out when it was last run, and summarise stats for all the required hours between then and now.

2. The point that you are running distributed stats is probably the reason you have leftovers - I would guess that central maintenance runs on the hour, but the system to transfer stats from distributed-mode "slaves" (they are not really DB slaves) fails, so the data doesn't arrive in time for central maintenance to see it. That's why you have to use the republish script to deal with the missing data.

There is no solution to this, but as a suggestion to make it happen less frequently:

1. Put good monitoring and alerting in place to ensure that the data transfer works; and

2. Delay when central maintenance runs to give the data transfer process more time to complete.

If you can, you can even hook #2 up to #1, and only run central maintenance once you know the data transfer is done. But this would have to all be your own custom code.... 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.




×
×
  • Create New...