Jump to content

Recommended Posts

Hi there

I've read few threads about scaling and distributed maintenance but they are all pretty old now. What's the current status - what's possible?

We would like to set up a horizontally scaled Revive adserver cluster, that would be able to serve up to 100k (or even 200k in the future) requests per second. On AWS. 

Would that be doable, you think?

Share this post


Link to post
Share on other sites

Hi there,

Yes it is possible. It's lacking documentation, but it works. And yes, you should be able to scale horizontally with those amount of numbers.

AWS is quite expensive though, if you're considering AWS, you might even consider outsourcing the hosting, and not have the technical challenges of running those numbers.

Share this post


Link to post
Share on other sites

Thanks for the answer. I understand that documentation is lacking - is there any at all?

If I'm right, the concept is still 1 database, multiple delivery machines, distributed maintenance. And seperate CDN for creatives delivery.

Or, is there any other major component to consider?

Share this post


Link to post
Share on other sites

Hi @murichej,

If you only have one database, you do not need distributed maintenance. With a one database, multiple delivery servers setup (with or without CDN), you only need to run maintenance on the one database.

Distributed statistics is for when you have multiple master database. Not recommended/supported.

Share this post


Link to post
Share on other sites

Thanks, that makes sense. I just don't get exactly, how will delivery engines create cached delivery rules based on the maintenance made every hour. How they communicate with main database?

So, my plan is now to:

- install admin instance & database through init.php
- install delivery engines through init-delivery.php
- put load balancer in front of delivery engines & enable sticky sessions
- set up hourly maintenance on main-admin instance

 Did I get the idea right?

Share this post


Link to post
Share on other sites

Just install it in the regular way, copy the codebase to the delivery boxes (you can skip the admin folder)

The delivery engines will need access to the database ! 🙂


You don't need sticky sessions

Share this post


Link to post
Share on other sites

Oh, now that sounds even simplier. Thanks a lot for your help. I'm going to try to establish a demo setup now.

Just one more question, will delivery engines connect to database for every ad request? That means that database still needs to be highly scalable, I guess.

Share this post


Link to post
Share on other sites

Hi again. I have managed to do a setup consisting of distributed (multiple) delivery engines, seperate admin and seperate database. Maintenance works fine, delivery works fine as well. If I shut down admin engine, delivery still works, which is okay.

But, if I shutdown database, delivery doesn't work anymore, even if the cache is already created on delivery engines. Digging into the issue I found that  'Banner Impression Logging Plugin' connects and writes an impression every time when ad is delivered (through avw.php). It updates table rv_data_bk_m every time.

And that means that I still have single point of failure which I can not scale. So, is there any option (or any other plugin) that temporarily stores impressions into the local storage or something, and then updates database every X minutes in example?

Share this post


Link to post
Share on other sites

I am on Amazon RDS (Aurora Serverless type) but it is a bottleneck I believe. 200k req per secod, I don't think Aurora Serverless can handle that. Mysql clusters could maybe work.

I am also thinking about writing the code to temporarily store impressions on delivery engines (in memcache or disk or similar) and then do a bulk transfer to revive database (to table rv_data_bkt_m) once every few mins maybe.

Do you think that's good idea? Anyone tried it already maybe?

 

Share this post


Link to post
Share on other sites

A plugin that caches delivery write data (e.g. into memcache, redis, etc.) would definitely be possible, if you believe that database clustering/failover, and vertical scaling won't meet your needs.

Don't forget that Revive Adserver also support PostgreSQL as well as MySQL - you may find performance differences there that assist with larger scale deployments.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.




×
×
  • Create New...