Use Shared Group Data for a live event system?
Question from a developer:
I’m implementing a live events system, which assigns players to “pods” of 100 people. These pods are my own implementation of leaderboards, and they are shared groups, with each key being a playerId and holding their info and score.
There is an index stored in title data for how many pods there are, and they are filled sequentially via CloudScript calls and have standardized names, like “pod[INDEX]".
I’m being careful to stay within the limits (# of keys, size of keys, etc) and since each pod has up to 100 players, there isn’t too many of them relative to the number of players.
But it just seems odd that I’m resorting to them for arbitrary data storage. I’m sorta doing the same thing for my implementation of friends lists.
Best Answer
Yes, I can see a few issues with that. Title Data isn't meant to be updated by player-generated actions (like people joining these pods). First, there will be scenarios with simultaneous attempts to update it, which will result in the count being wrong, since both attempts will have been to set it to N+1, when it should ultimately be N+2. Second, you'll have issues with players getting old values from the cache for a short period whenever it's updated, which will also cause them to do incorrect things. With the Shared Group Data, it is designed for a few players to share data, but if they're writing to the same keys, you'll have issues with people having their data stomped anytime there are simultaneous write attempts.
Edit (additional): The fundamental question to ask is, what are the update and access patterns for the data system you have in mind? Apart from things like Statistics, there are two types of data systems available in PlayFab:
1. Title/Publisher Data - Only infrequent updates, can be read by many/all players (sharded/cached).
2. User/Shared Group Data - Can have frequent updates, only read/write for a small number of users.
When planning your features, consider what will happen if you have high volumes of users - high concurrency, surges in new user registration, etc. If you have a need for a data type which is readable by large numbers of players and any of those scenarios could result in frequent updates, that would cause you problems.
The workaround for now would be to use an external data table. You can use http calls from Cloud Script, to it would be possible to use one with a Web API interface that way. AWS has this for DynamoDB, Azure has table storage, and mLabs makes a Web API interface available for their hosted MongoDB tables. We'll be providing a MongoDB interface in an upcoming release, as well.
We've used our own in-house backend for past games, but our hope in going with Playfab this time around was to avoid having to maintain our own system as we are a pretty small studio. So using an external (self-managed AWS) service kinda defeats the point.
More insight into my approach: I'm aware of the issue of non-atomic read/writes to the index in the title data. The index acts more as a "hint", and the cloudscript does the actual checking, so there is some room for staleness. For example, client A calls the cloudscript method, which reads the index and starts at "pod5", which is full. It moves on to "pod6", which is not full, so it adds PlayerA and updates the index to "6". Let's assume someone else in that same period writes the index back to 5. Player B calls, checks "pod5", sees it is full, checks "pod6", and either joins or keeps searching (up to a point) and then updates the index. So the index "generally" tracks with the latest unfilled pod.
The players in the pod read the entire shared group data as that essentially constitutes the leaderboard. When they need to update their scores for the live event (which I throttle somewhat so it doesn't happen more than say once a minute), they update only their key. I don't know what the underlying implementation for shared groups is; my hope is that the keys are separate and not part of a blob that is rewritten in full each time, which could definitely cause some data loss. Though again, the client is the authority on its score, so if it gets clobbered one time, it might eventually get through, and we always display the client's truth on their device.
The reason I'm resorting to this is, like Playfab's friend list feature, the leaderboard functionality is not nearly sufficient for my purposes and I imagine that of many commercial games, forcing me to essentially write my own layer on top of it. As far as I know, there are no player-agnostic data pools (that, unlike TitleData, scale with the number of players) available in Playfab at the moment aside from shared groups. So I've been using those. The Mongo DB access sounds like it would be a very valuable feature going forward to fill this niche, I'll be keeping a close eye on it.
I can't say I know how this will perform at scale. Theoretically it seems manageable based on what I described above, but we won't know until our game scales, if it ever even needs to.
My main question is, outside of your concerns about our own game's data integrity that you outlined above, how do you guys feel about this approach? Does it worry you, seem like an abuse? We are a professional studio and do intend to eventually pay (though the relationship between pricing models and usage limits remains a mystery at this point) but my understanding is the limits in place are as much about sustainability as they are monetization based on your blog post. Any insight is appreciated, sorry for the novel.
By the way: Another example use case I'm thinking for shared groups is clans, which seems obvious, except I would also probably use shared groups for indexing purposes. For example, we would probably want a way to show players some selection of clans they can join. We can't store that in title data as it would need scale with the number of players/clans (though we could keep a selection of the latest clans I suppose). So I would probably create a series of "clan_bucket[INDEX]" groups that would contain the clan ID's (themselves shared groups) as keys, and continually fill these buckets as clans are created and storing the index in title data similar to the leaderboard. Then I would randomly select a bucket in the known range to display to the user.
Going further, we would want these clans to participate in their own competitions with leaderboards, possibly with multiple skill tiers, at which point we would create additional shared group "pods" similar to the single player leaderboards above, with clans for keys.
Awesome - thanks for all the details! This sounds very well thought out, and may well work for you. Using the Here are my main thoughts/answers:
- No worries on the Shared Group Data keys - they are indeed distinct.
- Fully understood that you don't want to manage any backend tables (and that this is why you use us!) - just highlighting that for a space where we don't have a compatible system, that would be the option.
- I like the idea of the Pod numbering as suggestion, though I do still have a concern around what would happen if you were to get a surge of tons of players all at once. Corner case, I realize, but it's those corner cases that usually wind up tripping you up.
- For leaderboards, we will be providing additional arbitrary data per row soon, so it may well be that this would take care of some of your needs, there.
- For clans/guilds, we also plan to offer a built-in solution later on. Right now, my main concerns there would again be around data you want all users to be able to access. You could, for instance, have all the scores for the guild reported by the guild leader, changing who reports it and dropping the old leader to 0 in cases where guilds change hands. Ultimately, a truly robust clan/guild solution is dependent upon us building in support, as there's only so far you can go with Shared Group Data.
Also, we've updated the Limits page with all the prices for the upgrades, if that's the part of our pricing you were wondering about. Feel free to have a look and let us know what feedback you have.
