This document says that scorecards are generated at twelve hours and then every twenty-four hours thereafter:
https://docs.microsoft.com/en-us/gaming/playfab/features/analytics/experiments/
It has been over two days and the scorecard link keeps telling me to check back later.
Are they generated on a different schedule than the documentation states?
@Max Guernsey, IIIOK. We have seen your conversation with the engineering team on the Slack channel. Our community Slack workspace is still a great way for members of the community to stay connected and talk about how they’re using PlayFab in their games.
Besides, as our PlayFab Support Overview says, if you want a more formal way of requesting private support, we provide the assisted support channel for paid customers. You can click the link to learn about more details about PlayFab supports.
For what it's worth to the next person: They did eventually get this fixed.
We don’t have access to check customers’ Experiments page, but we are trying to reproduce this issue in the testing title. We will let you know if we have any progress.
Is there a query I can run in the data explorer that would help? Also, I have a free account... Could that be attention whether or not I get scorecards?
>> Is there a query I can run in the data explorer that would help?
Which types of query statements do you mean? There is no query statement that can be used to query the progress of generating the scorecards.
>> Also, I have a free account... Could that be attention whether or not I get scorecards?
It has nothing to do with whether it’s a free or a paid account. Every type of the account can generate and view the scorecards.
>> Any luck reproducing this?
We cannot reproduce it. In our testing, we kept an experiment in a running state for several days, the scorecard can be generated after about 12 hours. If we stop the experiment manually before it runs for 12 hours, the scorecard will not be generated. Did you stop the experiment in 12 hours?
1. I meant a KQL query. Since you can't see my data, I was offering to find what you needed to see and share it.
2. Okay. So we know it's not about account level.
3. I didn't stop the experiment. It's now been running for 4.5 days without a scorecard.
Here's what I've done that I can think might be mildly unusual:
* Stop other experiments (all short-lived test experiments).
* Delete other experiments.
* Dismiss a warning that I was manipulating a variable that was modified by another experiment (it wasn't true - there were no running experiments at the time).
* Make my total assignments 100% (30% control, 35% treatment A, 35% treatment B).
* Use a non-uniform distribution of populations (see the previous bullet).
* Manually test the experimentation by creating, running, and deleting another experiment shortly before.
* Save the experiment as a draft before I started it.
* Click "view scorecard" before the twelve-hour window.
* Set the duration of the experiment to 21 days.
* A while back, I upgraded to a paid account, then downgrade back to free - that involved manual effort on the part of someone at PlayFab.
Here's another data point. My experiment has 3 variants, including the control. Can that prevent me from seeing a scorecard?
Do you soft-delete experiments? I could have a deleted experiment by the same name as the one I'm running now...
@Max Guernsey, III We will inform our engineering team about this issue that your title's experiment scorecards cannot be generated on time to investigate it.
They actually contacted me on Slack, so no need. It's hard to keep track of the various communication channels. Especially since I don't get notifications from this site, anymore.
4 People are following this question.
