Best Practice for dealing with anomalous Automation cloud script timeout error
Every night, I run a script which zeroes out daily rewards from active players. During christmas, the script ( which usually takes 40ms to complete ) errored out for 750 users, apparently going over the 1 second time limit.
The question is, how should this be handled? I have found the documentation related to catching the error, but there is no clear example for forcing a retry. I also considered sending an email alert, but I don't really want to get 750 emails in my inbox.
Best Answer
What's the title ID? The execution time limit to Cloud Script should be 4.5 seconds for essential tier Titles and can be extended in paid tier.
To be honest, the design nature of PlayFab determines that there can be very few of failure cases when executing scheduled tasks and it can be caused by many factors. As long as it only occupies very little proportion, it should be normal. We will suggest developer to provide a support channel to players to submit issues for manual fixes.
There are many solutions, for example, collecting the reported PlayFab ID and add tags for them so that they will be categorized into the same segments. After some time (you can inform the players that the fix will come with next update), manually run the task again.
Another method (personally preferred) is to use PlayFab Explorer(located at [Game Manager] -> [Data] -> [Explorer]) to directly list the player IDs via advanced query in Kusto language. Filter the results for player_triggered_action_executed_cloudscript events. Please check "Publish results as PlayStream Event" for sceduled tasks.
Thank you.
The tagging into a group method sounds like a good way to catch it.
Perhaps run a checker script an hour later.
I am considering creating a player for the server to use as a data interface with the automated scripts : so if something errors out, it can set a flag on the serverData player.
Then, if I run a cloudscript on the single serverData player, I could get a single email alerting me of a problem.
Using internal titledata seems like what I want to access but to run Automated scripts one must always have a segment of players.
Is creating a serverPlayer for this purpose common practice?
This work around seems to be feasible. However, It will be more reasonable to configure a Rule in the Game Manager, to monitor the event "player_triggered_action_executed_cloudscript", where error message is contained in the context(set it as a filter). Then execute Cloud Script function or anything else you want.
Actively triggering things is always more efficient than passively and periodically running a checking task.
Great, thanks again. I'll proceed with this advice.
Side note : automation script time limit is 1 second.
Once more to clarify, I have an automated script that normally takes 30 millisecs to complete, it modifies a couple of playerdata variables for active players segment, and every so often some of them go over the title_exceeded_limit of 1 second.
I would prefer not to have the server mess with this, just handle it all in Playfab.
So, this seems to be the plan:
- catch error in script 1 to custom title event
- create rule to react to that event
- that rule calls cloudscript 2
- script 2 needs to aggregate the playerId somewhere - titleData?
- schedule cloudscript 3 to run a couple of hours later and process any slowpokes caught in the net
what if script 3 errors out as well though? Keep looping, 1, 2, 4 hour intervals
It seems pretty complex to do something that seems to be a commonplace occurence?
Normally, we won't recommend any backup plans for any scheduled tasks failure. It will be more efficient to handle this failure by players submitting a report for manual fixes. If there are massive issues for a task, you may submit a support ticket so that corresponding team will help you to identify the issue. The backup plans for scheduled task will not be effective as itself has the possibility to fail meanwhile, it is resource-wasting to implement another backup plan for a backup plan.
When you meet small parts of failure in a scheduled task very frequently. You should consider if the CCU has been increased and if the former functions can handle requests with the frequency increases. The APIs called in a Cloud Script function will still be counted in the Title Limits.
I beg to disagree. The script is light : it normally takes 30 milliseconds, and it is filtered to active player segment ( < 1000 per title currently ). We do not cause a heavy load, all our Limits are well within bounds. The failure is intermittent, but happens too often for comfort ( maybe once a week ).
handlers.resetDaily = function (args, context) {
var request = {
PlayFabId: currentPlayerId,
Data: {
playerDailyCoins: "0",
playerRedeemLink: "",
playerGoldenTicketRedeemLink: "",
playerGiftCard: "",
playerGiftCardLink: "",
}
}
server.UpdateUserData(request);
}
A failure of the script causes the player's client to see a prize they should not be getting ( and which they cannot redeem ), resulting in a bad user experience. Relying on a user email and manual adjustment of the playerData is quite the opposite of efficiency.
P.S. using an abbreviation like CCU is ambiguous. A search for CCU in docs results in 0 hits.
Thanks for the additional details. I will dig into this and discuss with the team.
