question

brendan avatar image
brendan asked

Missing logs for long-running server sessions

Question from a developer:

For my lobby server, it seems like when it runs for a long time, the logs do not get output and people cannot connect to the server (however, it is still running in PlayFab).

For example, Lobby ID DBA4C8AD40405D6E for Title ID 9092 does not have any logs, while a much shorter one (F420567D94C3B275) does.

The server flushes logs every few seconds or so, so I don't think it is an issue of logs not getting flushed to disk. Any ideas what might be causing this?

Custom Game Servers
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

·
brendan avatar image
brendan answered

Could you clarify the part about people connecting? Logs are captured when the server instance shuts down, so it's sounding like you're describing two separate issues, potentially.

For the log files, we have seen cases where servers that have extremely large log files may have issues with being able to copy them to S3 before the server is shut down. For example, you have one - Lobby ID D4A5512B807A909F where the log file is nearly 200MB, but was captured. How large do you believe the log file would have been for DBA4C8AD40405D6E, given the total runtime?

9 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Brent Batas (Lisk) avatar image Brent Batas (Lisk) commented ·

Sure - when this bug happens, the client is unable to connect to the lobby server (it just hangs at "Connecting to server..." in my UI and times out. It sounds like different issues, but I'm guessing it is just two different symptoms of the same issue so I thought to mention both.

The D4A5512B807A909F log is from a very old test version of the game and it's from the game server, rather than the lobby server. So it wouldn't be representative of the current logging for the lobby server. The lobby server just handles chat and matchmaking, so it wouldn't have nearly as much logs as the game server. That's why it seems reasonable for me to have the lobby server running for many hours.

A better comparison would be 81AB287A9AAF9F54 which is a lobby server that ran for 40 minutes.

Its logs are 250 KB. So just by rough estimate, I'd estimate the logs for the 8-hour lobby server (DBA4C8AD40405D6E) would be about 3 MB which seems pretty manageable.

0 Likes 0 ·
brendan avatar image brendan Brent Batas (Lisk) commented ·

Yes, that should be fine. So the issue is specifically that when the server gets in a state where players cannot connect to it, you terminate it and the logs never show up? Or if not, how is the server terminating?

If it's the former, that sounds like the game executable may be in a state where it's eating up all the available resources, blocking the operation that would normally copy the log file. The next time you have a server in this state, can you let us know so that we can log into it to have a look at what it's doing?

0 Likes 0 ·
Brent Batas (Lisk) avatar image Brent Batas (Lisk) brendan commented ·

That's exactly correct - I terminate it since I want to see the logs, but they don't show up.

I will let you know the next time this happens.

0 Likes 0 ·
Show more comments
Brent Batas (Lisk) avatar image Brent Batas (Lisk) commented ·

Thanks for checking. Good call - although I've had this bug happen before, and when I terminate it, logs don't get output (even if players can connect it). Just curious if you noticed anything odd, but it sounds like you didn't.

0 Likes 0 ·
brendan avatar image brendan Brent Batas (Lisk) commented ·

Not really. I saw no players connected, logged into the server, saw that it was running your exe, and when I looked back at the active games page, noticed that two players had joined.

0 Likes 0 ·
Brent Batas (Lisk) avatar image Brent Batas (Lisk) brendan commented ·

Is it possible to check if the server is outputting any log files to the output_files directory?

0 Likes 0 ·
Show more comments

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.