We want to store our game replays in playfab and then we would like to be able to query stored data using parameters like date or the players which played in the game and ...
Is there any PlayFab feature which I am not aware of and allows us to query the stored data?
I am not interested in maintaining my own index of the data using playstreams and a cloud script because we have no transactions here and some replay items might be missing from the index in event of a failure
As of right now no. As of future probably not.
-
The backbone of PlayFab's architecture is NoSql data storage. Typically with NoSql the burden is on the developer to manually handle querying based on relations.
-
In the current project, I am working on, we are going to be coming up against the same issue soon. Our current plan is to dump the game replays, game definitions into our own server, then load them into more appropriate databases for querying / analytics.
Actually, while arbitrary query for data files isn't likely, the core gameplay need that's described is a very real possibility going forward.
First, we will be adding custom data as an option for leaderboards in an upcoming sprint. Using that, you could either store some data with each player's leaderboard entry or (if you have large replay files) a reference to a Content file containing that info.
Alternately (though further out), we will be adding a UGC service, which would enable this sort of feature. For UGC, the need is to store arbitrary data, and then have "discovery" options for that data based upon things like most recent uploads, most popular (and being able to vote on them), as well as having reporting and take-down options.
The key with any feature is really to define the overall need from a high level, so that we can work with you to determine if there's a way to enable it, or a version of it, using existing tools or those in our near-term roadmap.
@Brendan so here are the details of what we need
For something like replays, we need to find all games of a specific user, best games depending on user votes and/or staff picks or based on some metadata in replay files.
Let's say replay data is a string of JSON commands and metadata is a json which needs to be queriable. DynamoDB's reading queries are good enough (I know you can not give direct DynamoDB access). We need some kind of filtering queries to decide best games should return those replays which ended the match in less than 2 minutes and with score greater than x and ...
Note that each replay is for more than 1 player and owned by multiple players. the data is write once and read many times after but for example if Alice and Eve play together and Bob wants to play EVE and see how they've been playing, he can search for both Alice and Eve games
If you're saving the player's game history in his User Data, you could query for a specific player that way. We do not provide a way to aggregate votes across players currently - that's what I was describing above. It may be available as part of our Guild/Team system we put in place later, but the design for that has not yet been finalized.
We do not provide open query on the data tables, as that's one of the most common things that causes problems for online titles - one bad query would slow performance for all players. And in a service that is used by many titles (such as ours), it would slow performance for many titles - something we prevent by having strictly controlled queries via our API calls.
For what you're describing, if you need to ship in the near future, I would recommend using an external data table. If you want to have some additional security on those calls via Cloud Script, you could use any Web API enabled database service, like Azure Table Storage, mLab MongoDB, or Amazon DynamoDB.
@Brendan yes we looked at using our own table for this and since my employer likes Amazon stuff more and is more familiar with it we are going the DynamoDB root.
About using GetUserData (I'm describing so you understand the case better) since a replay belongs to multiple players we either have to store them redundantly or maintain an index in each user's data and put replay in title data (I don't know if tables used for title data are good for storing many keys and having many requests or not). We will not go this root so i'm just saying hypothetically if we wanted to go this root, we had to do this.
And then there are issues like the fact that since there is no transactionality in DynamoDB then a monitor process should look for out of date indexes and other cases caused by failures.
I'm sure you are aware of this issue. I guess this is the reason that for example your friends system is unidirectional since you did not want to do a 2PC or at least did not want to do it at the time of implementing that.
Anyhow I understand that you can not give us query powers unless databases could be 1 table per tenant at least which is costly. You either had to use Mongo which had bad perf or 1 DynamoDB table per user so I can not bring down the title of the other users at least. I believe there are ways to solve this however which we can talk about in an email or something if you want.
I'm involved with scalable systems in ways unrelated to my current contract and specially with http://github.com/dotnet/orleans and thought much before on this.
Yes, Title Data is not the place for this, as it is designed to be a rarely-updated source of data available to all players (and if you tried to write to it for all players, you would rapidly consume all available keys).
You're exactly right on the reasons why a table per tenant isn't really viable. Interestingly, I was in the Xbox team when the Orleans project was first developed, and my own team (ATG, and then SSO) worked with the research team on possible ways to incorporate it into Xbox. I'd be happy to continue the discussion if you like at devrel@playfab.com.
2 People are following this question.
