question

omerkhalid avatar image
omerkhalid asked

Azure functions cloudscript vs JS cloudscript performance

Hi. We have recently built our architecture based on PlayFab and we chose implementing cloudscript using Azure functions due to obvious reasons. However, I have been wondering the performance of the two. Is it possible that JS cloudscript (which is the legacy cloudscript) might perform better as it might be hosted closer to the PlayFab servers or with the PlayFab servers while Azure function apps can be hosted anywhere and that might introduce a hop from playfab title to the azure function app?

CloudScript
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan avatar image
brendan answered

The only way to really answer this is to cover where we are with the service components, and where we're going.

Prior to Microsoft acquiring PlayFab, we built the service components in AWS. Since the acquisition, all new services are built in Azure, and we have been working for some time on the planning for moving the rest of the components into Azure, as well.

So today, some calls to PlayFab service endpoints from an Azure Functions Cloud Script will have faster ping times, and some will have slower ping times. Long term, all calls will be faster in Azure, since everything will be there.

As to average function times, both the legacy script service and Azure Functions can have cold start time - the time it takes for a server to get your script loaded and the script started, if a) there's been no script traffic lately (and that's on the order of seconds, not minutes), or b) you're at capacity on servers already running your script logic, and so need more. However, you can eliminate that pretty much entirely by using Azure Premium, which keeps an extra server running for you at all times. The same is not true in the legacy service. So to really test average response times, an apples-to-apples comparison is only possible by continuously running script calls - a one-time test is going to have too many other factors involved to derive any solid conclusions.

Yes, it is a fact that right now, you have to manage Azure Functions separately, in your own Azure account, and you "register" those scripts in PlayFab so that you can call them via API calls through the service. But my own preference is definitely Azure Functions for three main reasons:

1. More language support. Personally, I greatly prefer C#. Your mileage may vary, of course.

2. Far superior debugging. Legacy debugging is effectively just logging things from the code.

3. Better scalability. In addition to being able to use Premium, sudden, massive increases in traffic to your title can be managed in the legacy service, but you'd need to coordinate that with us ahead of time, so that we can pre-warm additional capacity. Simply put, Azure as a whole is orders of magnitude larger than just PlayFab, in terms of active compute usage. So the "buffer" available to absorb sudden increases is commensurately larger.

But then the bonus reason is that it's what we're investing in for the future. So pretty much all improvements to our scripting services will be made there going forward.

8 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

omerkhalid avatar image omerkhalid commented ·

Thanks for the detailed response Brendon.
Are there any recommendations as in which region to host our Azure Functions app?

Also, if I am correct, cold start works for the whole function app as one and not for each separate function in the app, right?

0 Likes 0 ·
brendan avatar image brendan omerkhalid commented ·

The core PlayFab services are all hosted in US West, so that would be the recommendation, in general.

A "Cold Start" is what happens when you have to spin up a new server with the script code for your title, either because there hasn't been script traffic for your title for a short period of time, or else because you've had an increase in usage/traffic that requires more servers. During that spin-up time, scripts could time out.

0 Likes 0 ·
omerkhalid avatar image omerkhalid commented ·

We are currently in development and our triggered actions which have a limit of 1 sec keep getting timed out even with simple operations such as fetching some basic stuff. What do you suggest for this as we cannot push some code on live just hoping that it would work better with more volume of calls without testing it first and we do not want to buy Azure Premium for the dev environment yet

0 Likes 0 ·
brendan avatar image brendan omerkhalid commented ·

The simplest thing is to have your tests spin up usage ahead of the "actual' tests. So, start performing logins and script calls, and then after a few seconds of that, start the actual test.

0 Likes 0 ·
omerkhalid avatar image omerkhalid brendan commented ·

I think my previous reply might not be well worded.
By scaling down I meant the idle time after cold start happens again.

0 Likes 0 ·
Show more comments
Show more comments
Jon avatar image Jon commented ·

Thanks for the insight.

From my tests the cold start cooldown is similar to AWS Lambda, just a little shorter.

Something like 20-30 minutes before your instance is turned down (AWS is more like 45-60 mins now)

@omerkhalid if you don't want to use premium, a work around is to have a scheduled call of your function every 5 mins to keep it warmed. You can have some code in your function to avoid executing the rest of the script if the request has `warmup = true` or something like that. The cost of doing that is pretty insignificant compared to premium

0 Likes 0 ·
Jon avatar image
Jon answered

I am interested in knowing this as well.

I did some tests in US-West-1 which should be closest to playfab servers

From my own tests it seems that the average function time is 800ms even without making any Playfab Server API calls. Adding a single playfab server API call like AddVirtualCurrency did not make much difference (maybe +50ms~)

If you check the cloudscript events generated by playfab the cloudscript execution time is like 150ms - 200ms even if you have 4 or 5 playfab server API calls.

Now I don't think the instrumentation is going to be the same for both methods, so the best test would be to do it from your game client and instrument it that way, but from my usage so far, it is definitely slower with azure functions. It may speed up if you have a high call volume if they have a function warmed, but there is a lot of overhead to just make 1 call every so often.

--

In addition, while azure function support is nice they are still missing some features compared to the old cloud script.

* being able to see errors in playfab event stream (unless you try/catch all your functions and return it in the response)

* logs / monitoring not integrated with playfab game manager dashboard (so you need to manage yet another dashboard, unless you use ANOTHER solution to combine it)

* sparse documentation and examples

* similar to previous point, but no best practices on the documentation means you spend a lot of time testing and refining to do things the best way

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.