I'm building up quite a chunk of custom CloudScript. Has anyone recommendations for getting it into a suitable testing harness?
I'm building up quite a chunk of custom CloudScript. Has anyone recommendations for getting it into a suitable testing harness?
What are you hoping for in a unit testing system for Cloud Script?
I started on one but had to switch off of it before it got to a solid feature set. (All I had was the ability to auto login to accounts and run sequences of Cloud Scripts with set arguments.)
I feel any long term solution to this has to be able to run offline just because of the need to
We have been talking internally about improved Cloud Script testing at a higher level. We are aware that testing Cloud Script is difficult, and we hope to improve that at some point.
For now, I can give you some of my own testing:
In my own testing, I create a test title, I create a real user, and I make real api calls. I'll repeat that I create an alternate title that won't affect a real game with potentially damaged user data or whatever. You should assume that CloudScript in development will sometimes break your user.
At the moment, that's the best suggestion I have for you. Be careful not to make too many api calls too quickly, or you'll get throttled. We will eventually have better options for developing and debugging Cloud Script, but for now do your best to test it however you can.
Thanks for the replies.
I'll preface this with the fact that I am pretty new to js. and in early development (but I have already managed to break other's work with insufficiently tested cloudscript).
My experience developing to date is that cloudscript creation is slow, error-prone and stone-age in debugging techniques. I'd appreciate more time efficiency, so I'm eagerly looking forward to the better options that you mention Paul.
I think a viable test strategy would be better with both offline and online testing.
I have some helper code independent of API calls and yet more that only has lightweight API dependency, that should be mockable with minimal effort. I could duplicate my cloudscript in a local environment to test the above functionality. I'm making the assumption that production cloudscript uses node.js, so using a common node stack including tools like Mocha, Sinon JS and Rewire might make sense. Even if my uploaded code is semantically correct, the debug activity loop is time consuming for typos or minor logic mistakes, and a unit test strategy could improve my productivity.
I also had the idea of a helper test title. I guess it wasn't as out there as I initially thought for online testing. These would be more integration than unit tests and would rely on a C# test framework to create calls into the system from a standalone program.
So I spent a while getting half of this strategy working. I used the straight C# PlayFab API to work in the most recent .NET and NUnit (rather than UUnit because my game uses Unity) to build a integration testing framework. It targets a duplicate test title with data copied over from the main title. It works nicely and has helped me pick up some script and economy data errors already.
Duplicating the economy data required writing a special application because I ran into a Catch 22 situation importing bundles and containers because their drop tables required the items, but their items required the drop tables. :-s
Now I'm starting on the pure js unit testing, but I'm blocked and not sure what is going on. I would like to use exactly the same script as I upload to the server. To do this I was planning on exporting functions that I would import into my tests using require. But when I try to use exports or module.exports the upload process complains that they are not defined and will not upload the file. Are module and exports explicity disabled in cloudscript? Or am I missing something else?
Yes, we're aware of the issue with duplicating titles (the chicken-and-egg with items, bundles/containers, and drop tables), and we'll be providing a tool later this year to make that process easier.
For Cloud Script, we're using a V8 JavaScript implementation, and I'm afraid we can't support modules just yet, though that is something we've got on our backlog for investigation.
I don't actually need modules on the server, all I could use right now is the system to not reject my cloudscript for exporting various items. The exports will be used to unit test on my own local environment.
My apologies - my own test wasn't correct. Right now, we do not support module.export. I've opened a work item for the Cloud Script owners to review the possibility of enabling this.
That's a shame. It makes unit testing cloudscript more of a manual hacker process for now. Hopefully when the time comes to enable that perhaps a way to enable mocking the server API calls can be folded in too please, because I currently haven't found a way to do that .
IMHO
{
Just to add random thoughts @Brendan
I've previously worked with other similar services to yours and their cloud scripts, It is a pain in the a** simply put.
I guess what is good is having support for a language like C# using Roslyn scripts or Azure WebJobs SDK which uses Roslyn under the hood and then C# code is much easier to test in that way, You can run the functions locally and API is same as playfab's server API for .NET there ideally and then finally at least in one provider there is a sane way to write cloud scripts. This is still a pain for many features due to the stateless nature of the cloud scripts but much more useful
}
Just my 2 cents, I'm not saying it is an easy thing to add in 2 days but this is a good thing for you at least long term.
5 People are following this question.