We have been using Playfab since 2019. Recently, the game was launched around 3 months ago, and there has been almost no timeout during the development period and even after launch. However, in the past few days, cloud script timeouts have been occurring very frequently.
The unique thing is that we haven't modified the cloud script at all in the last 3 weeks. So we expect it's not a script issue.
What should we do to reduce timeouts?
May I ask what's the title ID? I will dig into this issue. Please understand that it is common that with the player data (including items, entity data) increases, the API response time can grow due to the larger payload. If the runtime is already near the threshold, timeout could occur.
Title ID is 36A80.
Although player data has increased a bit over the three months since launch, I don't think it's changed that much compared to when the game launched.
On average, I remember that there were less than 10 times a day at most when the error was over 4k, but now, if you look at the cloud script processing time, the graph below appears to be full. It would be nice to be able to see past graphs.
The average execution time is usually 100~300ms, so it is still good when it goes well. But it seems that the situation over 4k ms occurs more frequently than before.
We have a very unusual phenomenon. We don't know for some reason, but the server has stabilized again!
As of 2021-10-08 15:39 UTC, the timeouts do not occur again as before. We're attaching a screenshot. We also attach an image of the server error count. Please note that the time on the lower line is +9 time changed locally.
Did you do any fix by any chance? If you didn't do any special work, could you find any changes between before and after that period?
We are very happy that the server is back to normal. We hope this will last. Thank you.
According to PlayFab Status, there was an incident of partial outage, which has been resolved.
However, I still suggest separate API calls If possible. Please feel free to let me know if you have any other question. Thanks for your understanding.
I have checked the backend of your title and there are timeout executions from time to time. Though the success rate seems to be fine for me, I notice that the timeout error occurs in the triggered “on Create Player” related functions.
The runtime limit for a triggered Cloud Script function is 1 second. To reduce this error, there are 2 workarounds. You have called server API GrantCharacterToUser and UpdateCharacterStatistics during this single execution, I suggest separating them into 2 Rules with independent Cloud Script Function. Since character events cannot be the triggered, you may use “entity_created” event as the trigger event.
Otherwise, you can implement queued Azure Function, which has much longer runtime limit.
There seems to be another problem other than the one you mentioned. The key is that the UpdateCharacterStatistics function is causing a problem. Both the OpenCharBox function and the Balance function call the UpdateCharacterStatistics function, but timeout occurs intermittently.
Until a few weeks ago, this was very rare. But now it seems to come out dozens of times an hour. I wonder why this problem suddenly occurred.
Below is a screenshot of the server log when the problem occurs. It would be appreciated if you fix the increase in the frequency of Timeout.
Hi, I have tried to reproduce this issue. During the tests, occasionally there is longer response time, but the success rate is fine.
As I have mentioned in the above reply, you may either use Queued Azure Function, or separate rules and using "entity_created" event for the other trigger.
Otherwise, you may also try to implement those APIs on the client as an initialization process instead of using automatic Rules.
You may be right if the timeout only occurs on entity_created. However, timeouts are occurring in all player_executed_cloudscript called from already created accounts.
This is a phenomenon that did not appear in the early stages of the game launch. For example, it appears in the OpenCharBox as in the attached screenshot.
This function is called when an already created character opens a gacha box. In addition to this function, it seems like it appears in all player_executed_cloudscripts that call UpdateCharacterStatistics with a very diverse distribution.
We will use Playfab in our next game, too. We would be very grateful if you could fix the timeout.
And we, the developers, aren't the only ones guessing this. Our users are complaining that the game has a lot of server lag compared to when it was launched. As for us, we have no choice but to respond by saying we are very sorry that the server is unstable. Thank you for your understanding.
2 People are following this question.
