question

Only saving periodically is a good first step, and having a throttle on how often the player can force a save is also important, so that all sounds good. However, I had a quick look at the title to see what the issue could be, and what I'm seeing is that your use of UpdateUserData is actually far higher than that, so there may be a bug in the client that you'll want to look into. Currently, what I'm seeing is that you're issuing around 10,000 API requests per DAU. This far exceeds our fair use limit, which is why the rate limiting eventually kicked in and throttled the title - specifically, to protect the service. This was reflected in the request rate limit exceeded errors you were getting before the throttle kicked in - they're harder to see in your error chart right now, as the retry logic is creating an overwhelming number of errors, but you can see them back showing up consistently prior to that.

We've made an adjustment that should bring your error rate down temporarily, but it's critical that you update the title to reduce the call frequency, to prevent this from kicking back in again. I strongly suspect that the retries are the main issue (so, when the throttling kicked in, this exacerbated the problem, as suddenly all clients started hammering the call), but the main things to check are:

1. Make sure that the client is definitely following your design for save frequency (only once a minute, and throttling any player force-save attempts).

2. For the re-try, use an exponential backoff on the call. If the client, on a failure, is immediately re-trying the API call, that's only going to make things worse for all players. Instead, it should back off for an increasing period of time on each retry. Something like 2 seconds, then 4, 16, etc. Since you're trying to auto-save every minute, if it hasn't succeeded by then, backing off until the next automatic save attempt would be the best thing to do.