Home Gitesh Portfolio Blog About Me Gallery Contact

XDB lagging to get reporting data

Our production website was getting more than 2 million hits per day. This started to cause delays in getting data into our Reporting DB. Data in prod reporting will be behind by 2 weeks. 

We spinned up another processing env to speed up the aggregation from mongo to reporting database. This made a difference but not alot. 

The we started to increase the number of threads that aggregates the data. We increased this thread to 200 per server and 100 contacts per server. This can be done in the aggregation.services.config file. Our SQL got really busy but now the server is able to get more than 2 million records in one day. 

Posted: 6/05/2022 1:05:46 a.m. by Gitesh Shah | with 0 comments