question

Brent Batas (Lisk) avatar image
Brent Batas (Lisk) asked

Scalability testing recommendations?

I'm working on rewriting my lobby server so that it scales to thousands of players per instance. One of the things I do is when a player logs on, I call GetPlayerCombinedInfo to pull their data. This is necessary as part of my login process so I can check things like whether they are banned, whether they have a game-in-progress they need to connect to, what their saved chat room was, what their skill level is for matchmaking, etc.

At some point, I'd like to stress-test my new lobby server with, say, 1,000 fake players logging in over a short period of time (1 minute or so).

I don't want to be hammering the PlayFab servers like crazy while testing, so I was thinking of having a "fake" GetPlayerCombinedInfo call that just calls Thread.Sleep(1000) then returns some test data. Then once things are working with fake data, I'll try performing the actual GetPlayerCombinedInfo calls to see how it performs in actuality.

Is this a good way to test? I'm pretty new to testing things for scalability, so I would appreciate any advice/tips.

The main thing I'm trying to prevent is having things work *in theory*, then when we do a big marketing push, have them fail under heavy load.

apistest
10 |1200
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

·
brendan avatar image
brendan answered

Sure, using known-quantity test data (that matches the schema of the real data, of course) is a good way to minimize the number of factors in your test. That way, you can test for all the conditions you want to have in your main flow, and you can have cases where you leave off one or another piece of data or have them returned with crazy values in order to have good bounds testing on your code.

It's akin to black box vs white box testing. With the real service, you've got the additional complexity of not having direct control over those inputs, so you have more to check to debug any issues. By starting with known inputs, you can more rapidly iterate on the core logic before turning on the real data.

10 |1200
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.