GetTitleData sometimes returns old results

Question

question

zeke asked May 17, '18

GetTitleData sometimes returns old results

I'm noticing that if I change a TitleData field in the PlayFab dashboard, calls to GetTitleData will randomly return either the old value or the new value for a few minutes. This means that if a client fetches title data to see if anything has changed, they might mistakenly see that the field has changed FROM the new value TO the old value.

Is this expected behavior? It's not noted here.

In our particular application we have some global auth controls set in our title data and these change semi-regularly (obviously auth isn't only client-side but the client makes a best effort to read & display the current auth state of the world in an informative and user-friendly way, and this behavior disrupts that).

apis

10 |1200

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Answer 1 · 2018-05-17T20:56:12Z

1807605288 answered May 17, '18

I have updated the guide. Apologies for missing that. We've answered this question multiple times in previous forum posts, and in Zendesk tickets, but somehow the documentation was never updated.

https://api.playfab.com/docs/tutorials/landing-content/using-title-data

This is by design. As the new paragraph in that documentation states, Title Data is cached for upto 15 minutes, and repeated requests may not reflect recent changes. There is simply no way to instantaneously update hundreds of machines if Title Data is updated. It simply takes time, and that's unavoidable.

[EDIT: pulling this answer up from a deeper location in the thread]

Title Entity Objects can give you a "best of both worlds" answer.

Title Entity Objects do the same caching as TitleData, but they include a ProfileVersion number. So you can read that number and determine if you are reading an old object, or a new one.

https://api.playfab.com/documentation/entity/method/GetObjects

Fetch the title Entity objects, and check the ProfileVersion number to determine if you are reading an updated value or not.

This is about the best scalable answer that you can get in cloud computing.

4

10 |1200

Attachments: Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

zeke commented · May 17, 2018 at 09:23 PM

Thanks for the quick reply & doc update Paul!

The surprising part isn't the latency, it's the fact that reads aren't monotonic, so clients that request title data more than once can see values traveling backward in time (and have no idea that that's what they're seeing!) Lots of distributed-ish databases guarantee monotonic reads, so it was just unexpected (and oh god horrible to debug) that PlayFab doesn't.

If the Playfab API doesn't make those consistency guarantees then I'd like to be able to do something better on the client side than just have every client randomly "flutter" between stale and correct values for 15 minutes.

Is there a way to see the timestamp for when a value was updated? Then my client can just discard any "updates" that have a stale timestamp.

If I update two fields in a single SetTitleData call, are those updates atomic? (will they reach the same PlayFab server at the same time)? The SetTitleData docs don't specify. If so, then I can add a separate timestamp field for every field and just make sure to always update the value and timestamp in a single call. (Otherwise, I need to turn the field values into JSON blobs to include a timestamp along with the value).

0 ·

1807605288 ♦ zeke commented · May 20, 2018 at 02:52 AM

I our case, you get this effect because there's so many servers:
Get: New Server A cashes Data 1
Set: (You update data)
Get: New Server B caches Data 2
Set: (You update data)
Get: New Server C caches Data 3

You make 3 more sequential calls, and get:
Get: C returns 3
Get: B returns 2
Get: A returns 1

If you update data a bunch in a short time-frame, your requests will bounce around all the servers handling your traffic, and potentially get very strange results.

Updates will happen much more as you expect them to. They're making writes to the database, and not a local cache. So the most recent update/write that the database has processed will (eventually) be the "real" version.

If you do a bunch of writes from cloud script, somewhat simultaneously, they'll each be writing FROM a different cached version of title-data, and they'll each write independently of one another, and each one of them will have no real way of knowing if they data they wrote is the data that actually stuck. (In other words, don't write titledata from cloud script, you have essentially no idea what was written, when, or whether it stayed that way).

0 ·

zeke 1807605288 ♦ commented · May 20, 2018 at 06:51 PM

Thanks Paul. I understand the problem. Do you know if any of the solutions I suggested in my other reply actually work? The API docs aren't very clear about the details of the consistency guarantees.

Regardless, I've talked it over with the team and this behavior makes title data mostly unusable for us (and not worth the work to cleverly work around). We're just going to switch to an AWS database. When we read data from AWS their API lets us specify whether we want eventual consistency (which is what playfab has) or strong consistency (which is what we want - never returning old results).

0 ·

Show more comments

question