I am a bit lost when it comes to analytics and report generation.I have added a set of custom events related to player behavior (the kind of character that has been selected, duration of a match, etc.) The events are correctly stored and I can perform Kusto queries to count them, filter them...
Now, I need to provide the game designers with dashboards that allow them to explore statistics built on these events.
However, my understanding is that the events are only kept for 30 days. I hesitate to invest time into building advanced Kusto queries, because it will be impossible to execute queries for long-time (> 1 month) trends in the game.
I have added an automated S3 backup that saves all events to a bucket, and I have checked that the JSON files can correctly be retrieved from these archives, but working only from this data will not scale really well when the number of players will (hopefully) increase.
A possible option would be to automatically run queries that perform some sort of data compression by turning the raw events into statistics, as a first step, then pushing these "summary events" into a database that I would manage on my own.
My question: in case the "compute summary of events" described above is a good solution, are there available mechanisms to automatically execute kusto queries that perform counts, averages... and export them to some endpoint, with production-grade quality? (automatic retry in case of failure, monitoring, alerts...)?Is this something that I should setup an external server / cloudscript functions to trigger?
More generally, if you have some interesting resources to share about the typical architecture of an analytics reporting system with lots of users, I'd be very grateful.
I realize my question is broad, but any advice, even anecdotal, is welcome.
Thank you very much in advance!