Pro Tier limits and the entity data

Question

question

Hjalmar Vikström asked Sep 24, '18

Pro Tier limits and the entity data

Hi! I'm Hjalmar, programmer at 10 Chambers Collective, working on a game called GTFO coming out for PC and Consoles.

We are looking into going to the Pro tier and want more information. I've spent a good time searching the website, the forums and the playfab control panel. But I've only found some scattered information like this one: https://community.playfab.com/questions/13455/limits-for-pro-tier.html regarding the tier limits.

Anyways, I got a bit scared by the Data Limits on the free tier and as I said I couldn't find the complete information.. So here are my questions:

1) I would like to know all the limits in the Pro tier, as found in Settings > Limits. How do I get that info?

2) Regarding the data limits, what does "the same" mean in "Number of times the same player data value may be updated..."? Is it a unique key/value per for a unique user that can't be updated too often, or is it a global limit for all players updating the same key/value pair? Or something else?

3) Also data limit: which data are we talking about in this limit "Inventory item data value size"? Is it the custom data on the catalog items or the much more limited custom data on the inventory item instances?

4) This one is a bit longer, about the new entity data API:

I was super happy with the player data, I could add anything I wanted there, as a simple but extreme test I added 500 posts with a good chunk of JSON with no trouble (not added in the same post..), so I felt sure I could add any custom data I would need there as long as I don't update them too often (in our case we would update when starting the game and when the player performs a custom update and saves, almost never during gameplay.)

But now I read about the entity system and it feels a bit weird, it seems like with the entity system you want me to group all my data to a big blob of JSON or whatever and then update the whole thing as an entity object or an entity file, right?

How could it be more efficient to update one big file of 100K instead of posting one update (playerData key) of with 1K of data amongst 100 possible keys? This makes me worried... or am I missing something?

5) If we design our data around player data, can I be sure you are going to fully support it down the line or is it basically deprecated?

Thanks!

/Hjalmar

entities limits data pricing

10 |1200

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 1 · 2018-09-24T19:48:49Z

brendan answered Sep 24, '18

Hi Hjalmar - good to meet you! The link you referenced is indeed accurate, and is a pretty comprehensive list of the differences. Specific to your questions:

1. The data limits are the last couple of items in the post you linked. Is there something else you're looking for? Not all limits have increases. For example, those that are key to preventing a title from causing performance issues (like the number of keys per request in our older data model) are not increased, apart from special arrangements in the Enterprise tier where we work with the developer/publisher to get the details of their needs so that we can come up with a custom contract that covers the additional resources that'll be dedicated to their titles. In the Enterprise model, we can do far more custom things.

2. A single key for a single player.

3. Custom data stored on an individual inventory item instance. A catalog item can have 1K of data. An instance of an item on a player can have up to 5 keys of up 100 bytes each (since players can have a great many items).

4. The Entity model provides two distinct data systems - one for small JSON blobs, and one for larger files. This was as a result of observing how titles were using our data systems to date, and coming up with optimized models for each of the two major use cases. In short, the small data packages (Entity Objects) would be significantly more expensive than the Entity Files, if they were the same size.

5. We have over 2K live titles (and growing), and there's no universe in which it would make sense for us to try to get them all to update for a change we make. That's why we have a firm "no breaking changes" policy. If we deprecate a feature, it will no longer be available to new titles, but it will never stop working for existing titles that are using it.

10 |1200

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 2 · 2018-09-25T00:22:43Z

Hjalmar Vikström answered Sep 25, '18

Hi and thanks for the quick answer!

1) My main concern is custom data limit on item instances and the player data limits. Our players can update the custom data for their item instances when they mod them.

I currently can't fit the customized data in 100 bytes so I work around the data limit on instanced items by creating a player data key for each item instance that needs to have customized data and store the custom data on the player data instead.

Any suggestions for other ways of storing around 1K custom data per item instance per player?

I will try to minimize the JSON, even though it's really great to be able to read the data for debugging purposes I guess we can't afford it.

Another concern is the 1K custom data limit for the catalog items. We have plans for giving more abstract items that is just a holder for the custom data, and some of the things we have discussed could require more than 1K.

4) Alright, my main problem is that I find it really attractive to have multiple small-ish keys where I can drop player specific things like settings, global progression data, level specific progression data, temporary custom stats, item data if needed etc.

And having that reduced to "you have 3 small objects and 1 big file" is a really weird update for me.

Maybe I'm thinking about it backwards, in that case I'd love suggestions on how else to think =)

5) Ok, sounds good, I just want to know that we won't be locked out of a feature we are heavily relying on even though we are not releasing the game yet.

Another question: we are also going to rely on title data delivering live data updates to the players, so we need to get at least one or two title data values when we start our game, and they probably will be close to the 20K limit. Do you see any problems for that to scale?

Thanks.

1

10 |1200

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan commented · Sep 25, 2018 at 01:21 AM

Got it:

1. Item instances have a limit of 5 key value pairs of up to 100 bytes each, so it's really 500. The reason for the limit is because players can have a great many inventory item instances. If you need more data space, it might be best to use player data or entity files. Other things you can do is to write the data in a schematized manner, so that you don't have to have an explicit key for each value. So instead of {"Version": 3, "Weapon": "Sword"}, you would just have "3;1" (with 1 being your enum value for "Sword").

4. Player data wasn't designed for small KVPs, which is why we added the Entity Object system (which is). My main recommendation would be to aggregate the things you're tracking into fewer, larger packets of information. Just from a data OTA perspective, it's far more efficient (packet overhead on SSL is a minimum of 40 bytes, so if you're sending lots of tiny bits of data, a huge percentage of the bandwidth used is overhead).

Title Data: Nope, that's what Title Data is designed for.

1 ·

Answer 3 · 2018-09-25T17:19:56Z

Hjalmar Vikström answered Sep 25, '18

Thanks!

1) I think this has been my main problem, I'm just serializing a class and storing that JSON-blob, even a small class ends up being 1K easily.

I will try using just an ID-array and see how lean I can get. It get's less/not readable so I lose some of the beauty of JSON in my mind, but I want this to scale properly so I'll probably have to <=)

4) If you don't mind me asking, what kind of data is the player data system designed for? what is the optimal size of a value to be efficient?

I hear what you are saying about overhead and I will keep that in mind. I think my main problem with the entity system is that it is meant for small KVPs but I only have 3 keys for title_player_account which in this case would be the interesting entity for me.

This means that I don't have much room for extra data down the line unless I want to compress all my data into oblivion and be really sensitive to version changes of our data formats.

If I could keep the data a little more separate, I don't need to worry about the player settings data format changing interfering with his level progression etc. Does this make sense to you?

Additional questions:

6) if I put it this way: what is the typical use case for a data structure saved on an entity object and one example of data to put in the entity file? Not for my game but for the most typical games where this system fits.

Are the entity files recommended for everything that is bigger than 500 bytes?

Thank you for your patience and answers, I really want to understand this so that we make the right decisions now instead of having to change it later.

8

10 |1200

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

brendan commented · Sep 25, 2018 at 05:41 PM

4) Player data was the original data system from when we launched PlayFab. It is simply a string data store per key. It was intended simply as a basic data store for any type of data. Hence, the reason we moved to the Entity model, once we'd seen how titles were most commonly using it. Anything under 1KB in that system still uses as much resource as 1KB, so that'd be the minimum to go with. But to be clear, the Entity Object system provides for 5 1KB objects in the Indie/Pro/Enterprise tiers.

6) That varies wildly between games. Some are using Entity Objects to store their game save state, as they have very simple definitions of their game state. More complex games save state as an Entity File. Objects might also be a sub-section of the key elements of the player state (what is equipped, what interface settings have they selected, etc.).

Entity Files are intended for data you can't fit in Entity Objects. In general, I'd say 1KB or more, but the Essentials tier has a limit of 500 bytes per Object, so in that tier, yes, 500 bytes.

1 ·

Hjalmar Vikström brendan commented · Sep 25, 2018 at 06:21 PM

Alright, so If I keep our data un-optimized and around 1K, player data might not be too bad of a fit.

Depends on the load I guess. Let's say our game is successful, a couple of hundred thousand players buy it, and on startup all players download a 1K player data for each inventory item they keep, ranging from 1 to 200-ish. Would you be worried?

My other options being compressing the data and trying fit it on the custom data of the item instance (which we also get on startup) or putting all of the item-data in a entity file.

0 ·

brendan Hjalmar Vikström commented · Sep 25, 2018 at 06:52 PM

We're completely unconcerned about the number of players, as we scale for any number. What concerns us is usage per player. So if each player is reading/writing a small number of keys from/to player data infrequently, there would be no issues.

0 ·

Show more comments

question