question

Martin L avatar image
Martin L asked

recommendations for unit testing cloud script

I'm building up quite a chunk of custom CloudScript. Has anyone recommendations for getting it into a suitable testing harness?

CloudScript
1 comment
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Joshua Strunk avatar image Joshua Strunk commented ·

What are you hoping for in a unit testing system for Cloud Script?

I started on one but had to switch off of it before it got to a solid feature set. (All I had was the ability to auto login to accounts and run sequences of Cloud Scripts with set arguments.)

I feel any long term solution to this has to be able to run offline just because of the need to

  1. Constantly reset player/character/item data
  2. Constantly check player/character/item data
  3. The time increase per test added when running against the actual servers would be astronomical.
0 Likes 0 ·
1807605288 avatar image
1807605288 answered

We have been talking internally about improved Cloud Script testing at a higher level. We are aware that testing Cloud Script is difficult, and we hope to improve that at some point.

For now, I can give you some of my own testing:

https://github.com/PlayFab/SDKGenerator/blob/master/JenkinsConsoleUtility/jcuSrc/Testing/CloudScriptTests.cs

In my own testing, I create a test title, I create a real user, and I make real api calls. I'll repeat that I create an alternate title that won't affect a real game with potentially damaged user data or whatever. You should assume that CloudScript in development will sometimes break your user.

At the moment, that's the best suggestion I have for you. Be careful not to make too many api calls too quickly, or you'll get throttled. We will eventually have better options for developing and debugging Cloud Script, but for now do your best to test it however you can.

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Martin L avatar image
Martin L answered

Thanks for the replies.

I'll preface this with the fact that I am pretty new to js. and in early development (but I have already managed to break other's work with insufficiently tested cloudscript).

My experience developing to date is that cloudscript creation is slow, error-prone and stone-age in debugging techniques. I'd appreciate more time efficiency, so I'm eagerly looking forward to the better options that you mention Paul.

I think a viable test strategy would be better with both offline and online testing.

I have some helper code independent of API calls and yet more that only has lightweight API dependency, that should be mockable with minimal effort. I could duplicate my cloudscript in a local environment to test the above functionality. I'm making the assumption that production cloudscript uses node.js, so using a common node stack including tools like Mocha, Sinon JS and Rewire might make sense. Even if my uploaded code is semantically correct, the debug activity loop is time consuming for typos or minor logic mistakes, and a unit test strategy could improve my productivity.

I also had the idea of a helper test title. I guess it wasn't as out there as I initially thought for online testing. These would be more integration than unit tests and would rely on a C# test framework to create calls into the system from a standalone program.

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Martin L avatar image
Martin L answered

So I spent a while getting half of this strategy working. I used the straight C# PlayFab API to work in the most recent .NET and NUnit (rather than UUnit because my game uses Unity) to build a integration testing framework. It targets a duplicate test title with data copied over from the main title. It works nicely and has helped me pick up some script and economy data errors already.

Duplicating the economy data required writing a special application because I ran into a Catch 22 situation importing bundles and containers because their drop tables required the items, but their items required the drop tables. :-s

Now I'm starting on the pure js unit testing, but I'm blocked and not sure what is going on. I would like to use exactly the same script as I upload to the server. To do this I was planning on exporting functions that I would import into my tests using require. But when I try to use exports or module.exports the upload process complains that they are not defined and will not upload the file. Are module and exports explicity disabled in cloudscript? Or am I missing something else?

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan avatar image
brendan answered

Yes, we're aware of the issue with duplicating titles (the chicken-and-egg with items, bundles/containers, and drop tables), and we'll be providing a tool later this year to make that process easier.

For Cloud Script, we're using a V8 JavaScript implementation, and I'm afraid we can't support modules just yet, though that is something we've got on our backlog for investigation.

7 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Martin L avatar image Martin L commented ·

I don't actually need modules on the server, all I could use right now is the system to not reject my cloudscript for exporting various items. The exports will be used to unit test on my own local environment.

0 Likes 0 ·
brendan avatar image brendan Martin L commented ·

My apologies - my own test wasn't correct. Right now, we do not support module.export. I've opened a work item for the Cloud Script owners to review the possibility of enabling this.

0 Likes 0 ·
brendan avatar image brendan brendan commented ·

Update - our Cloud Script is based on the latest V8 JavaScript engine, but it is not built on Node.js. Since that is a Node.js specific feature, it's not something we can support in our Cloud Script currently.

0 Likes 0 ·
Martin L avatar image Martin L commented ·

That's a shame. It makes unit testing cloudscript more of a manual hacker process for now. Hopefully when the time comes to enable that perhaps a way to enable mocking the server API calls can be folded in too please, because I currently haven't found a way to do that .

0 Likes 0 ·
brendan avatar image brendan Martin L commented ·

We are indeed working to provide more debugging and testing options as soon as we can. We'll definitely be updating with a post about this as soon as it's available.

0 Likes 0 ·
amazdeh avatar image amazdeh commented ·

IMHO

{

Just to add random thoughts @Brendan
I've previously worked with other similar services to yours and their cloud scripts, It is a pain in the a** simply put.
I guess what is good is having support for a language like C# using Roslyn scripts or Azure WebJobs SDK which uses Roslyn under the hood and then C# code is much easier to test in that way, You can run the functions locally and API is same as playfab's server API for .NET there ideally and then finally at least in one provider there is a sane way to write cloud scripts. This is still a pain for many features due to the stateless nature of the cloud scripts but much more useful

}

Just my 2 cents, I'm not saying it is an easy thing to add in 2 days but this is a good thing for you at least long term.

0 Likes 0 ·
brendan avatar image brendan amazdeh commented ·

Thanks for the feedback! We do want to add a C# version of Cloud Script in the future, but it's going to be a little while until we can get to that. I have added your feedback to our internal tracking item, though.

0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.