A recent build caused our servers to be stuck on Initializing for 10 minutes, then restart.
We've tried reverting to a previous version but the result is the same.
Is there a way to diagnose what's going on?
Are there startup logs available somewhere?
Thanks for the info, guys. We had some remaining issues from yesterday's update that caused the state where game modes were briefly missing for titles. What happened is that folks who edited their game modes while they were in that state had two sets of data that didn't agree, so that was causing some issues. We've been working through re-touching each of those titles to fix this. If all your game modes are back to the complete set for your title, we should be done with your title. If you're still seeing issues after that, can you let us know?
Unfortunately still happens for us with the latest build version 0.13.0.147.
The game mode is auto-added by our CI OK, but server instances are constantly being spawned & cycle between Initializing / Unavailable every 10 mins or so (bit worried it might accrue charging for each instance cycle but will leave it on for a few in case you want to take a look at its active behavior EDIT: leaving just US-East going for that reason)
Example Unavailable server: 18.207.118.21 2600:1f18:70d:5100:b817:fd85:82aa:e335 ID: i-00b4b293d0cf888d1 USEast
One potential cause idea could be if the AddServerBuildRequest backend was somehow having a similar issue to the Game Mode one and leaving a duplicate default invalid server build config around? It all looks correct in the dashboard UI, but if the DB has another (invalid executable path) default config that gets fed to the server on startup, that might cause it to look for the wrong executable path (gameserver.exe instead of Clone Drone in the Danger Zone.exe which is our game's configured path) which could explain builds going Unavailable only some % of the time
So the issue is that while you had removed and re-set your modes, there was still bad data leftover from the other day. Now that we've completed the cleanup work, can you re-test?
Correct -- tested and seems to be working perfectly now! Thanks very much for the cleanup work and helping sort this out, this is a big relief!
Hey Brendan, was running into this issue yesterday too as I mentioned in the ticket... builds were going Unavailable -> Initializing over and over, and I did notice the game mode was missing randomly. I must have edited it and gotten into that bad state. Anyway today I uploaded a build and it worked fine so I guess it's all good now.
Some more details (I'm working with Erik on the title E79E):
The servers cycles between "Initializing" and "Unavailable" every 10-15 minutes. We've disabled all regions for the builds that won't start (0.13.0.140, 0.13.0.145 and 0.13.0.146) just in case it's charging for server time each time it is auto-starting.
We also tried uploading a previous version, known working build 0.13.0.140 which also didn't start up, and tried clearing our CI system's cache & got the same issue.
I also tried manually building & running 0.13.0.146 on my PC, and the server starts up on my computer OK. EDIT: tried manually zipping & uploading that to PlayFab and that seems to launch OK. Hmm...
EDIT: Now finding some servers of the existing known working build are giving status Unavailable. Example build 0.12.2.3 (live game) Server ID: i-02d806532e453f1d6
EDIT: Now finding that re-enabling regions for previously non-launching builds (without changing uploaded build file at all) creates both a working Running server and Unavailable one (launched 3 seconds apart)
Hi there --
I have the same problem. Build is stuck in Initialized. How can we debug that? And how can we do any clean up if necessary (re-answer from @Brendan)? Any help with that would be much appreciated.
BTW -- the server works fine locally!
Thanks.
@brendan
I'm curious about the answer to the original question. Is there a way to see logs for what's happening as a build deploys?
Same problem. Since yesterday every build, on US or European server is stuck initialized...
5 People are following this question.
