Jump to content

horizontal scaling


Recommended Posts

Hi there

I've read few threads about scaling and distributed maintenance but they are all pretty old now. What's the current status - what's possible?

We would like to set up a horizontally scaled Revive adserver cluster, that would be able to serve up to 100k (or even 200k in the future) requests per second. On AWS. 

Would that be doable, you think?

Link to comment
Share on other sites

Hi there,

Yes it is possible. It's lacking documentation, but it works. And yes, you should be able to scale horizontally with those amount of numbers.

AWS is quite expensive though, if you're considering AWS, you might even consider outsourcing the hosting, and not have the technical challenges of running those numbers.

Link to comment
Share on other sites

Thanks for the answer. I understand that documentation is lacking - is there any at all?

If I'm right, the concept is still 1 database, multiple delivery machines, distributed maintenance. And seperate CDN for creatives delivery.

Or, is there any other major component to consider?

Link to comment
Share on other sites

Thanks, that makes sense. I just don't get exactly, how will delivery engines create cached delivery rules based on the maintenance made every hour. How they communicate with main database?

So, my plan is now to:

- install admin instance & database through init.php
- install delivery engines through init-delivery.php
- put load balancer in front of delivery engines & enable sticky sessions
- set up hourly maintenance on main-admin instance

 Did I get the idea right?

Link to comment
Share on other sites

Hi again. I have managed to do a setup consisting of distributed (multiple) delivery engines, seperate admin and seperate database. Maintenance works fine, delivery works fine as well. If I shut down admin engine, delivery still works, which is okay.

But, if I shutdown database, delivery doesn't work anymore, even if the cache is already created on delivery engines. Digging into the issue I found that  'Banner Impression Logging Plugin' connects and writes an impression every time when ad is delivered (through avw.php). It updates table rv_data_bk_m every time.

And that means that I still have single point of failure which I can not scale. So, is there any option (or any other plugin) that temporarily stores impressions into the local storage or something, and then updates database every X minutes in example?

Link to comment
Share on other sites

I am on Amazon RDS (Aurora Serverless type) but it is a bottleneck I believe. 200k req per secod, I don't think Aurora Serverless can handle that. Mysql clusters could maybe work.

I am also thinking about writing the code to temporarily store impressions on delivery engines (in memcache or disk or similar) and then do a bulk transfer to revive database (to table rv_data_bkt_m) once every few mins maybe.

Do you think that's good idea? Anyone tried it already maybe?

 

Link to comment
Share on other sites

A plugin that caches delivery write data (e.g. into memcache, redis, etc.) would definitely be possible, if you believe that database clustering/failover, and vertical scaling won't meet your needs.

Don't forget that Revive Adserver also support PostgreSQL as well as MySQL - you may find performance differences there that assist with larger scale deployments.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.



×
×
  • Create New...