question

James McGhee avatar image
James McGhee asked

750 free compute hours not enough for basic testing or are we doing something wrong?

So we have been using PlayFab for a awhile now while in development and can clearly state that 750 free compute hours isn't enough for a month.... or we are doing something stupid

Here is what we are doing ... 2 devs that's all that is testing and developing a client server app we are working on. We set things up with the F2v2 I think it is ... forget where we got that recomendation but that is the one we have been using. Setting it up for 2 servers per machine so we can go in and out of various sessions without hitting the no more servers message

Typically we hit 700-ish about mid way through the month and I have to belive we are doing something wrong here. I mean massive majority of that time is idle in just 400 real hours we burn through right at 700 compute hourse. Now I get the whole standby system means there can be 2 servers up at a time as we connect and drop out of a given server but any time I check there are 2 on standby when I configured for 1 ... this I assume is because we have 2 servers per 1 VM.

For note that means we are burning at the moment about 1.75 compute hours per 1 real-time hour ... and that is not 1 real time hour of testing tha tis 1 real time hour as in within on 25 hour day we see about 43 compute hours gone.

Why do we have servers up all the time?

So we can do rapid play testing together .... we are widly distributed team working at all crazy hours so a local server doesn't do anything for us, only having a server up 9-5 doesn't do anything for us. We need to be able to pop on and off at will, try some things and move on. This means that we more or less always have a build deployed.

So what are some things we could do to meet the following work requirements

1) Basicly always need a cloud hosted server up for play testing with the team

2) Need to keep it at or under the 750 a month... defently not double that :) ... this isn't about avoiding costs its about understanding how this very low use can equate nearly double time on the clock

3) Need this hosted server to be as close to what would be in production as possible ... so the server allocation aspect and all that not interested in just tossing up a VM on Azure and calling it good enough ... we moved past that when we started up with PlayFab services

Any guidance would be appreciated.

multiplayer
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan avatar image
brendan answered

The simple answer is that most teams don't have your requirement #1 - that you need to have servers running constantly throughout your work day. Our free hours are not designed for that - they're meant to give you enough time to do validation testing on your builds, which normally only requires a few hours a day, at most. Quite a few teams only require a few hours in any given week for that.

Your model sounds much more similar to a game built at a mid-to-large developer, and which is in the final phases of development. So, you have a broad segment of your team (potentially including an independent test team) doing testing, requiring you to have servers running for long hours, every day. In that model, yes, I would definitely expect you to exceed the free hours.

It also sound like you have some question on how server hours accumulate ("its about understanding how this very low use can equate nearly double time on the clock"). The answer is that it's the number of cores times the number of running VMs. Since you have your build set to 2 servers per VM, if you have your Standby level set to 2, that would mean that you have one server running even when you're not testing, since your standby level of non-0 (which is normal - when you're not testing, just set it to 0), and as soon as you launch 1 server, you only have 1 still available in Standby (again, since your configuration is 2 per VM), so another VM has to spin up and be ready.

Also, please bear in mind that these are Core hours. An F2sv2 is a 2 core machine, so one VM uses 2 Core hours per real-time hour. not 1. An F4sv2 would use 4 Core hours per real-time hour.

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

James McGhee avatar image
James McGhee answered

Thanks


We will play with the config options and try to keep it as low as possible. You noted that this would be in line for a larger team but I would say its also common for a widly distributed team. When you cant simply stand up a local server for moment to moment testing you depend on a server in the cloud ... it doesn't need to be fancy, fast or even friendly to use :) it does however need to be there more or less all the time and at least act like its using the services your building ontop of.

In the past we would just stand up a VM on Azure or similar and use it, that worked well, didn't cost much at all but of course that isn't using PlayFab multiplayer, that doens't let us work through the allocation system or with the entity objects, etc. that we need to be working with at the same time so its not really an option for this project.

I will say we are not a big team nor is this project big its that we are a distributed team developing activily ontop of your services so we need a place to deploy a server that can actually use your services not just validation testing but experamentation as well. I would expect other small indie teams with a handful of members scattered all over the world would similarly want a "dev" server they could work against that acted like a real PlayFab multiplayer server and could be always on. ... maybe a nice to add feature later on?

2 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan avatar image brendan commented ·

With distributed teams, the normal way to approach this would be to have limited windows for "everybody's testing" times that work well across your geos, rather than just leave the VMs running all the time so that whenever someone wants to test they can. If you feel that level of convenience is a requirement, then you should definitely expect to exceed the free hours each month.

But it's worth noting that:

a) It only takes a few minutes to provision a VM and download your build to it - not something that you would want a user to have to deal with, but not unreasonable to ask of your dev team, if the scenario is one developer needing an hour or so to validate what they've been working on.

b) A common misconception is that if you start a server and test for a few minutes, you're billed for an hour. We don't bill in full-hour increments. Our runtime granularity is one minute. So if three different engineers spin up VMs for five minutes each, that's fifteen minutes, not three hours.

0 Likes 0 ·
James McGhee avatar image James McGhee brendan commented ·

Yep I expect we will probably go with the free feeding approch ... have people deploy there build, use it then delete it when done. Will add a bit of overhad here and there but its servicable. Most little play sessions are a half hour or so here and there with less frequent couple hour sessions on occasion so that should keep the time down nicely.

All that said your rates arn't bad it wasn't a sticker shock that spured the question more of us thinking of that 750 as a "team dev server budget"

0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.