question

sickshark avatar image
sickshark asked

Latency for Free tier

Hello!
What is the worst latency for the free tier? and the average one?
The first is more important though. Thanks.

data
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan avatar image
brendan answered

Latency refers to the total time from when you send the request to when you get the response. That's a combination of the processing time for the call, and the time it takes for the packets to be routed.

For the former, you can see the average processing times for all calls in the Daily API Usage Details Report in your titles. The time will vary from call to call, depending on the work required, but the average time for processing of calls is normally in the tens of milliseconds range for most calls. The obvious exception is Cloud Script, which is arbitrary script you write which can do a combination of things, including make other API calls. How long that takes depends upon the complexity of what you're doing and any calls made from the script (particularly to third-party services, using http.request). So technically, the "worst" possible processing time would be a Cloud Script execution, for a script where you're making external API calls, and the script eventually times out due to the third party not responding. Since the maximum Cloud Script execution time in our free tier would be 4.5 seconds currently (for a script run from a Client or Server API call), that would the worst case.

In the free tier specifically, a title that is exceeding a safe limit on usage (excessive calls per player) may be moved to a "quarantine" server vertical we manage, to prevent causing any issues to other titles. In that case, response times may grow significantly, as that vertical does not have the same level of resources as the primary PlayFab service vertical. So even non-Cloud Script calls could have significant latency in that scenario, though the worst case would be unlikely to exceed a couple of seconds.

Now, for the latter (time to route packets), that's going to depend upon the location of the caller and their Internet Service Provider (ISP). The main PlayFab service currently operates from the west coast of the US, so more distant locations will naturally have longer packet latency. The ISP (and regional network in general) is also a critical part of this, as older infrastructure or governmental firewalls such as the Great Firewall can introduce significant delays.

But all that said, the PlayFab service is almost entirely a set of Web API calls. What specifically are you trying to determine? What is your scenario (presumably) that requires very low latency? If you're working on a game that requires frequent updates and low latency, the recommendation would be to use a custom game server (whether hosted by us or elsewhere), so that you minimize the roundtrip time to the player, and can exchange/update data at a high rate.

1 comment
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sickshark avatar image sickshark commented ·

I want to make a game with 4players competing to each other with an highscore asynchronous. I need to stay in the free tier because I don't know if my gains will be enough to cover playfab expenses. I want to have a the best update time available for this tier so I would like to have an update of the score every 5 seconds pushed to all 4 players. More this time is short better it is. With 5 seconds means 12 update per minute per player, this should be in the range of free tier right?

Besides this, I was worried to the total roundtrip, the real delay from the start of a call from the player to its possible reply. Like from the most far player to your servers, how much time does it take to go and back? Worst case and average scenario?

In free tier do you provide a cdn like photon to short this time or not? also, I read in January you will launch your multiplayer system, will you make available to free tier as well under certain limits? I hope so!! So we don't have to go to photon anymore...

Thank you !

0 Likes 0 ·
brendan avatar image
brendan answered

Answering here, since it's a bit too lengthy to be a comment:

In terms of your design, I would recommend either using Photon or else a custom game server. While updating data at that rate wouldn't be an issue, you're talking about distributing the scores between all the players. So either you have to have the clients directly sending the data to each other, or a server tracking the scores sending them out. Otherwise, your plan would have to include the clients additionally querying the leaderboard every 5 seconds, which definitely would be excessive and would result in throttling on your title.

Put another way: what you're trying to build is a realtime game system in which players are all updated with information about all other players frequently. That does require either a relay service, like Photon, or a custom game server. Which you choose will depend upon how secure it needs to be. If you need high security, you need custom game servers.

In terms of the total roundtrip, that's specifically what I was explaining above. There are too many factors involved for there to be a specific average that would apply to all players, regardless of location. Using a relay service or custom game server will help to minimize that time, with custom game servers giving you the best update time (and allow for even higher frequencies of updates, if you want to go there). In that model, it should always be down to barely more than the point-to-point latency time - which should always be sub-second, unless you're trying to talk to players behind the Great Firewall from outside China or vice-versa.

For your other questions:

CDN - We do have a CDN service, but that's not relevant to your question. A CDN is used for distribution of content, not for exchange of state data between clients. But for completeness, no, CDNs aren't free, as the amount of usage is going to vary wildly per client, across different games. In our case, the cost is purely usage based.

Photon (Realtime/PUN) - We do not currently provide a relay service similar to Photon Realtime or PUN.

Multiplayer server hosting - We do provide a small amount of server time for testing, but anything past that (which a live title would need) is also usage based. And again, total usage per player will vary wildly across different types of games. How much of the game state you want to manage server-side will determine how many players you can host per server.

Both CDN and server hosting are charged as close to our cost as possible. They're not designed to be revenue streams.

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sickshark avatar image
sickshark answered

Dead Brendan, so your multiplayer with Azure is still under development? Nothing with a tutorial or avalaible to the Free tier?

I dont want to make a real-time game. I want to know the threashold for the updates in order to mantain a little service up.

I remeber there is a tutorial using shared data for games similar to Clash Royale alike. Where everything is asynchronus.

What are the options? Leaderboard and shared data?

If i want to use the former, what would be the update rate to not make it throttle? 7 seconds?

And what is the same update rate for the latter one in order to make it right?

if possibile i would like to stay in the Free tier at any cost, even changing the gameplay or include a higher latency if that is necessary.

Finally, you said there are too many factors influencing the end to end connection, but i believe other developers must have asked this to you since is the crucial for the gameplay! If one developer make a car game must to know what is the delay there you provide! So lets imagine we use the free tier, all the players Will always be connected to the server in US? so player in India will have the slowest delay and how much this would be? You dont have a reference figure? Like 4 seconds to go and other 4 to come back?

thanks

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan avatar image
brendan answered

Multiplayer Servers 2.0 is in public preview:

https://blog.playfab.com/blog/public-preview-of-multiplayer-servers-2-0

In terms of usage, you should think of it this way: A few requests per minute, for most operations (including Shared Group Data read/write) and averaged over the full game session, isn't a problem. But calls that return a lot of disparate information, like leaderboards and profiles, are commensurately more expensive to operate (by comparison to something like write or read a player data key/value pair), and so should be called less frequently. Once a minute (again, averaged over the life of the game session) would be fine.

Shared Group Data could be another option, but bear in mind that if you have multiple players writing to it at the same time, they're going to stomp each others' updates - that's why we recommend only using it for turn-based games.

In general, no, people don't ask us about latency because we don't control it. It's controlled by the factors I outlined above. But for purposes of that calculation, yes, the core API calls for PlayFab are all going to servers in the western US currently. You can get an idea of the general latency from a number of sites that provide realtime ping times for Azure and AWS (ex: https://azurespeedtest.azurewebsites.net/). In general, those times will be sub-second (the highest I usually see in testing is around 400ms). But the very nature of the Internet and how packet routing works makes it impossible to say what the maximum time could be, because technically it's infinity for any case where your packets are dropped somewhere in the route. If they're not dropped, I've seen cases where the latency of a call that had to cross the Great Firewall was a bit over two seconds.

But there is no "you can send exactly N of this API call, M of this one, etc., etc." guide. To be candid, this is because ideally games should be optimizing their call patterns as much as possible. This is particularly true for mobile games, as high usage is going to drain the player's battery faster. But also, bear in mind that what we're trying to do is keep our costs low, and keep a free tier. If we had a ton of free tier titles trying to max out how much of the service they can use, that would inevitably force us to either remove the free tier entirely or limit it in ways that would be too restrictive for many game types.

Ultimately, you need to decide what the gameplay experience is that you're trying to create. If you can describe that, we can advise you on the best approaches for the backend of that game. Game designs that require a high frequency of regular communications to the client usually need to have custom game servers in order to optimize that part of the backend, or if they're okay with not be highly secure and just want to exchange data between players quickly, a relay service like Photon Realtime.

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sickshark avatar image
sickshark answered

Alright. Ideally i would like to have an update every seconds where each player writes its current score into the shared data plus if he sent an attack to another player encoded with integers or strings. Like 1 2 3 could mean player 1 send attack type 2 to player 3.

No other information are exchanged. just these regular updates of the highscore and eventual attacks to send in queue to the other player via cloud script.

Now, for me there is no problem if I update this to each 7 seconds. I know is a big delay but I can't afford more from my freemium game where I also help charities. Another player will receive the attack 7/8 second later. I know.

I want to know what is the best rate I can achieve, is not about max out, is about to know the limits of the plan, I would ask the same if I had another tier. What is the rate that doesn't make it throttle and gives me.the best result ?an update each 4 seconds? 6 seconds?

Game is for 4 players simoultanously with this delay yeah.. also you stated shared data update stomp to each others. You mean they cue right?so if two update comes at the same time one is executed after the other right?

They do not overwrite badly hope not.

Yeah also I will do my best to make it delay tolerant.

Thank you :)

1 comment
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sickshark avatar image sickshark commented ·

*the attack information is simply encoded plus the current player score into the shared data update. Both are written in there, in the data structure. Very simple.

0 Likes 0 ·
brendan avatar image
brendan answered

There is no way to "send" data to other players via Cloud Script. In this design, each player would write to a separate Key in the Shared Group Data. No, these writes are not queued. Using a separate Key per player is how you would avoid overwriting each other. Each player would also need to read the SGD at a regular rate to accomplish what you have in mind. But you would want the write to be the player's last 3 moves, at least (depends upon how long you want to let them stay in the session if they miss one or more updates). That way, if a client fails to get an update (say the player is driving through a tunnel, for example), that information will still be available when he gets around to re-trying the call.

The way to think about most standard cloud-based data tables is that each "hit" (read or write) to a KVP increases your "heat" (it's the read/write capacity of the table technically - a single KVP being hit a lot is more costly than many hits to a spread of KVPs, but in your case you have to read all of them every time). So realistically, you should design for your writes and reads to each be no more frequent than about once every ten seconds. If all the players are trying to write to a single KVP, we'd have to re-evaluate that, so I'd stick with a separate KVP each.

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sickshark avatar image
sickshark answered

Ok Brendan so no more than 10 seconds update. So, the client calls the cloudscript every 12 seconds, the cloudscript check the validity of the update and updates the shared group data, right? And immidiately reply to the player that sent the write with the updated state,and to the other players as well. Correct?

But when you say the title will throttle what will actually mean? The server will crash? Or will ignore certain requests of read/write?

Is possible to have the same design but using the leaderboard to store these data? With that can I achieve a slightly better update rate?

Finally, we are making a charity game for helping the environment and poor people hit by chatastrophies. It's our first project so we don't even know if we will rise up enough to give help, not even to cover our expenses already invested. I hope this long delay solution will be enough pretty to work, so if we gain we will definitely invest in the multiplayer. If you know any better free solution let me know.

Also, let's say I see I need better resources, can I upgrade live into another tier without any losses? And also when paying is relative to the use or is a fixed quota? Since I see 99 EUR per month per indie. Pay per use is better to me but I don't understand that 99 dollars.

Thank you

7 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sickshark avatar image sickshark commented ·

My worry( I believe of most devs) is that I may be ending in the future to a situation where I have many many active users but none of them is converted to a payer so I can't sustain the servers cost.

what do you suggest to devs in the these case?

0 Likes 0 ·
brendan avatar image brendan sickshark commented ·

The answer to that question would be a fairly hefty tome on game monetization methods, Live Game Operations (LiveOps), and marketing across the industry. If your game is not designed for monetization, whether through up-front purchase of the game or IAP, it's a bad idea to build in features that have ongoing costs. If you have designed for IAP (up-front purchase takes care of itself) and you're not converting players, you need to figure out why they're not converting and fix that. The possible reasons for that and solutions are extensive, but the most important thing is being able to have all the data about what your players are - and aren't - doing, which is what the PlayStream event data is for.

0 Likes 0 ·
sickshark avatar image sickshark commented ·

One last thing: you said 10 seconds update rate for shared group data.
But where do you get this info from? From the Limit on shared group data shown in the game manager i see only these below. It comes simply from your knowledge or you read it somewhere or deduced from somewhere?

Shared group data updates per request
Number of shared group data key/value pairs which may be updated in a single request.
Shared group data value size
Size of a shared group data value, measured in UTF-8 encoded bytes.
0 Likes 0 ·
brendan avatar image brendan sickshark commented ·

It's an estimate based upon assumptions of average usage, using the specifics of the scenario you described. It's not something you could easily calculate, as it's largely based upon experiential knowledge from the thousands of titles that have gone live in PlayFab.

0 Likes 0 ·
sickshark avatar image sickshark brendan commented ·

Yes but would be great if you state such limitations somewhere! for example I don't see any limitations for the rate of the cloudscript. I see only limit on its size. For example one will assume I can even call it every half seconds! Can I ? Where can i see such limits? It's not nice if there are other limits which are not clearly stated.

0 Likes 0 ·
Show more comments
brendan avatar image brendan commented ·

Specifically:

The Cloud Script will return to the caller specifically what you put in the return value.

Throttling, in this context, would mean that calls would be blocked - it cannot cause a server to crash.

No, leaderboards are a more CPU intensive service. If anything, your read rate would have to be lower, causing more of a delay.

I'm not aware of any free service that would provide more resources for realtime updating of data. I'm very sympathetic to your goals, but it's not possible for us to provide free server hosting. The more expensive our free tier gets, the more likely it becomes that we'll no longer be able to offer it, so we work hard on making sure folks know the best practices for backend design.

Moving to the Indie tier would not change any data or player accounts in your title. It would increase the tier limits, which you can review here:

https://community.playfab.com/questions/13455/limits-for-pro-tier.html

But please note that read/write rates are not a limit that we can increase, outside the Enterprise tier.

0 Likes 0 ·
sickshark avatar image
sickshark answered

I am asking if the architecture I am explaining is correct:
-each of the four players send an update every 10/12 seconds that consists in calling the cloudscript
-the cloud script will check if the update is legit and if so it will write inside the shared group data.
-After this it will also send a push update of the state to the other players.

Can you confirm?

For throttling, blocking means that if I call too many times the API or the cloud script the requests will be dropped or queued?

The link you sent me is for PRO limits not for Enterprise Nor for INDIE. Where can i see the limits for these? Also showing the limits of frequency of calls of cloudscript or shared group.

what do you mean? are you planning to cancel the free tier from the night to the morning? I hope not, thats not very reliable. As I wrote elsewhere: Gamespark offers a fantastic business plan, they give you everything for free included te multiplayer till the 100000MAU and if one grows more than the dev pays. So litttle developer are not worried if the game will not succeed. it give more relax and no pain to re-code if too much intensive. you code once, if the game succeed one jest pays ! please can you do like them ? instead of pushing us to make playfab+photon free?

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan avatar image
brendan answered

As I said above, the part that isn't correct is the "push" of state to the other players. Cloud Script will return data to the calling player. There is no "push" to the other players. The order of operations for your script would be:

1. Read the Shared Group Data.

2. Update the calling player's data in the SGD, if it's valid.

3. Return the SGD KVPs of the other players' move info to the calling player.

Throttling/blocking means that the calls are rejected - they are not queued.

The Indie tier and Pro tier have the same limits, specifically because when you hit 100K MAU in the Indie tier, you auto-convert to Pro. It's just a low-cost flat-rate version of Pro that doesn't include private support or SLA. I've updated that post to help clarify that.

While we have automated throttling to prevent highly abusive calls (usually due to bugs), those are not part of our stated limits, as a) we may adjust them over time, and b) there is no set rate limit that is valid for all titles, for the reasons stated above. Please use the limits I've advised in this thread for your specific scenario.

I have no idea what you're referring to about cancelling "from the night to the morning". We have no plan to get rid of the free tier, as we feel it is highly valuable - it allows indies to ship games at no cost, so that they can work out their monetization before moving to the Pro or Indie tier and getting access to the full range of LiveOps tools to maximize their title's potential. The point I'm making is that we cannot provide things like free custom game server hosting or support excessive call rates in the free tier, as it would make it too expensive to operate.

And again, what you are referring to in their "multiplayer" offer is their relay service. Not custom game server hosting. If you don't need the security of having a server running your code to manage the game state, a relay service would be your best bet, yes. PlayFab does not currently provide a relay service. We integrated with Photon, which does provide a relay service (Photon Realtime and PUN). We do not control Photon's pricing model, as they're a completely separate company.

5 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

sickshark avatar image sickshark commented ·

When you write "2. Update the calling player's data in the SGD, if it's valid."
Yes the problem is exactly here. I want perform a Cloudscript check to see if the client data just received is valid and legit and let the server to write into the shared group which is safer! How can I do this? the cloudscript is meant for this, isn't it like it is explained in te tutorials?

Yes but if you don't state the limit for the calls how can I know if my title is throttling or going to throttle?
Lets say I stay in free tier, doing 10/12 update rate. it doesn't matter if I have 100 or 1000000 players to do that right? It will not throttle, can you guarantee this? and also as long I respect all the limits of usage I am allowed to stay in the free tier no matter my MAU, correct?

0 Likes 0 ·
brendan avatar image brendan sickshark commented ·

Cloud Script is arbitrary JavaScript you write, and which lives on our service. You can see an example on the Automation->Cloud Script tab, in revision 1 (which is automatically loaded onto all new titles). That example includes showing how to read/write data.

If you're doing what I describe - calling a Cloud Script handler once per 10 seconds per player, which simply does one read of a Shared Group Data, some simple validation logic which doesn't require any additional service (or other Web API) calls, and one write to the player's key in the SGD, you're not going to run into any throttling issues.

Specifically, there are two ways you'd run into issues: 1. Exceeding the "high" limit rates. Those are hard limits built into the service. 2. Exceeding fair use limits. Those are indeed undefined as there is no simple way to define them. Simply put, if your title is using the service in a way that is not supported, we will contact you to let you know what needs to change. In the case of the free tier, we may have to put your title in our quarantine servers while you make the fix, which can result in more error responses, depending on what your title is doing.

0 Likes 0 ·
brendan avatar image brendan sickshark commented ·

And yes, currently (as of January, 2019), our free tier has no MAU limits. You are never forced to change tiers due to usage, though we may have to throttle titles that are in violation of our fair use policy.

0 Likes 0 ·
sickshark avatar image sickshark commented ·

So the architecture will be like:
A client calls the Cloudscript sending him some parameters
The cloud script validates the action and writes into the SGD.
The other players read the updated SGD. ( Can you confirm that is not possible to push the update to clients from the cloudscript instead?)

Is this correct?

Secondly, let's say in the future i want to increase the frequency of calls to the cloudscript to 2seconds rate.
Can I do it? How? am I forced to use the multiplayer server service in that case or switching to pro tier is enough?

Thirdly, how can i monitor if a title is about to throttle?

Finally, I believe that almost anygame have a server side logic to prevent cheatings. And you are saying that Photon and Gamespark do not have such functionality? this means that only you or Amazon have a little cloudscript/server logic to check this? You don't know any other company offering this service out there for indie developer except you at Playfab?

Thanks :)

0 Likes 0 ·
brendan avatar image brendan sickshark commented ·

Yes, as I said above, the Cloud Script is only going to return the results to the caller. You cannot "push" data to others using it.

And also as stated above, no, if you need a high frequency of data updates (higher than we've discussed in the design I described above), you would need to use a custom game server or relay service, as that would be far more efficient. We could conceivable work out an Enterprise agreement with you where you pay for the additional server costs for the added load, but frankly, it'll be more cost effective for you to move to a relay or dedicated server solution.

The thing to look for in terms of throttling would be an increased error rate on the title. You can use the API graph on your dashboard to check for that.

As to Photon and Gamesparks, no, that's not what I'm saying. Both of them do offer server-hosted logic services. The specific services you asked about above are relay services which do not provide for server-side logic. The services to look at are, respectively, "Bolt" and "Cloud Code".

0 Likes 0 ·
sickshark avatar image
sickshark answered

I am confused from what you wrote instead in this other thread:

Let's clarify this, correct me if I am wrong:

you say Photon and GameSpark DO have custom server service but from Bolt page https://www.photonengine.com/en-US/BOLT, BOLT says"

    • Photon Cloud does not offer to host Unity Instances. Possible solutions are Gamelift (by Amazon), Multiplay (by Unity) or custom solutions." What does it mean exactly? It does have a custom server logic or not ? I can upload other types of code but not unity?Then how is different from PUN?
    • CloudeCode instead seems just like Playfab Cloudscript.. so it's not a custom server service...

    I need to perform a check on the legitimacy of the players actions in order to prevent cheating.
    Potentially to a rate of 4/5 seconds. What would be the best option to do this and at which cost?


    Does a relay service allows me to do this, yes or not? If yes, with Bolt but that requires gamelift or the deprecated Multiplay from Unity right? I want to do the less effort to integrate so many systems together..

    Can I do it with the Gamespark Cloudcode since compared to playfab doesnt have rate limits?

    Or If I need instead a custom game server, can I do it with your original custom server?
    if yes at which costs?

    What would be the difference with your new custom server service instead? the absence of a matchmaker? this means that If I want to use your old custom server service I would also need PUN to make the matchmaking as well?

    Finally, 0.008 per MAU means that if I have a million player I will pay 8000 euros per months?! :O and what happens if none of these convert into paying players?


    10 |1200

    Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

    Write an Answer

    Hint: Notify or tag a user in this post by typing @username.

    Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.