question

drallcom3 avatar image
drallcom3 asked

[Unity][Android] API compression does not work

Edit: Tests with Fiddler show that the SDK claims it's umcompressed, but the response is clearly compressed. Technically a bug, but any user can ignore this bug.

API compression work in the editor (2020.3.8 LTS Platform Android), but not on the device.

  • Using UnityWebRequest. PlayFabSDK: UnitySDK-2.108.210511.
  • Headers, URL, payload is the same in both.
  • Request compression works fine on both (makes it even weirder!).
  • Response is compressed properly in the editor, but not on the device!
  • Editor respose bytes: 31 & 139
  • Device response bytes: 123 & 34
  • No errors or warnings at any time. Response is fine on the device, just not compressed.

I've tried various things, but none helped. I even tried other Unity versions and other projects, but the result was the same. It's a mystery to me. Nothing indicates that the server even know it's a device making the call, yet he delivers different responses.

Try it for yourself with an APK. Sample project can be provided upon request.

Here's what I did in PlayFabUnityHttp.cs to find out about the issue (the rest of the project is a simple LoginWithCustomIDRequest and nothing more):

public void MakeApiCall(object reqContainerObj)
        {
            CallRequestContainer reqContainer = (CallRequestContainer)reqContainerObj;
            reqContainer.RequestHeaders["Content-Type"] = "application/json";
            Debug.Log("Payload before compressing: " + System.Text.Encoding.UTF8.GetString(reqContainer.Payload));

            Debug.Log("Compression: "+ PlayFabSettings.CompressApiData);
#if !UNITY_WSA && !UNITY_WP8 && !UNITY_WEBGL
            if (PlayFabSettings.CompressApiData)
            {
                reqContainer.RequestHeaders["Content-Encoding"] = "gzip"; //GZIP
                reqContainer.RequestHeaders["X-Accept-Encoding"] = "gzip"; // UpperLower case doesn't change anything

                using (var stream = new MemoryStream())
                {
                    using (var zipstream = new Ionic.Zlib.GZipStream(stream, Ionic.Zlib.CompressionMode.Compress,
                        Ionic.Zlib.CompressionLevel.BestCompression))
                    {
                        zipstream.Write(reqContainer.Payload, 0, reqContainer.Payload.Length);
                    }
                    reqContainer.Payload = stream.ToArray();
                }
            }
#endif

            Debug.Log("Payload after compressing: " + System.Text.Encoding.UTF8.GetString(reqContainer.Payload));

            Debug.Log("FullUrl: "+reqContainer.FullUrl);
            Debug.Log("ApiEndpoint: " + reqContainer.ApiEndpoint);
            string headers = "";
            foreach(var h in reqContainer.RequestHeaders)
            {
                headers += h.Key + ": " + h.Value + "; ";
            }
            Debug.Log(headers);


            // Start the www corouting to Post, and get a response or error which is then passed to the callbacks.
            PlayFabHttp.instance.StartCoroutine(Post(reqContainer));
        }

        private IEnumerator Post(CallRequestContainer reqContainer)
        {
#if PLAYFAB_REQUEST_TIMING
            var stopwatch = System.Diagnostics.Stopwatch.StartNew();
            var startTime = DateTime.UtcNow;
#endif

            var www = new UnityWebRequest(reqContainer.FullUrl)
            {
                uploadHandler = new UploadHandlerRaw(reqContainer.Payload),
                downloadHandler = new DownloadHandlerBuffer(),
                method = "POST"
            };

            foreach (var headerPair in reqContainer.RequestHeaders)
            {
                if (!string.IsNullOrEmpty(headerPair.Key) && !string.IsNullOrEmpty(headerPair.Value))
                    www.SetRequestHeader(headerPair.Key, headerPair.Value);
                else
                    Debug.LogWarning("Null header: " + headerPair.Key + " = " + headerPair.Value);
            }

#if UNITY_2017_2_OR_NEWER
            yield return www.SendWebRequest();
#else
            yield return www.Send();
#endif

#if PLAYFAB_REQUEST_TIMING
            stopwatch.Stop();
            var timing = new PlayFabHttp.RequestTiming {
                StartTimeUtc = startTime,
                ApiEndpoint = reqContainer.ApiEndpoint,
                WorkerRequestMs = (int)stopwatch.ElapsedMilliseconds,
                MainThreadRequestMs = (int)stopwatch.ElapsedMilliseconds
            };
            PlayFabHttp.SendRequestTiming(timing);
#endif
            Debug.Log("Result:");
            if (!string.IsNullOrEmpty(www.error))
            {
                OnError(www.error, reqContainer);
            }
            else
            {
                try
                {
                    byte[] responseBytes = www.downloadHandler.data;
                    Debug.Log(www.url);
                    Debug.Log("Payload before decompressing: "+System.Text.Encoding.UTF8.GetString(responseBytes));
                    bool isGzipCompressed = responseBytes != null && responseBytes[0] == 31 && responseBytes[1] == 139;
                    Debug.Log("responseBytes[0]=" + responseBytes[0] + ", responseBytes[1]=" + responseBytes[1]);
                    Debug.Log("isGzipCompressed: " + isGzipCompressed);
                    string responseText = "Unexpected error: cannot decompress GZIP stream.";
                    if (!isGzipCompressed && responseBytes != null)
                        responseText = System.Text.Encoding.UTF8.GetString(responseBytes, 0, responseBytes.Length);

                    Debug.Log("responseText before decompressing: " + responseText);
#if !UNITY_WSA && !UNITY_WP8 && !UNITY_WEBGL
                    if (isGzipCompressed)
                    {
                        Debug.Log("Decompressing!");
                        var stream = new MemoryStream(responseBytes);
                        using (var gZipStream = new Ionic.Zlib.GZipStream(stream, Ionic.Zlib.CompressionMode.Decompress, false))
                        {
                            var buffer = new byte[4096];
                            using (var output = new MemoryStream())
                            {
                                int read;
                                while ((read = gZipStream.Read(buffer, 0, buffer.Length)) > 0)
                                    output.Write(buffer, 0, read);
                                output.Seek(0, SeekOrigin.Begin);
                                var streamReader = new StreamReader(output);
                                var jsonResponse = streamReader.ReadToEnd();
                                //Debug.Log(jsonResponse);
                                Debug.Log("Payload after decompressing: " + jsonResponse);
                                OnResponse(jsonResponse, reqContainer);
                                //Debug.Log("Successful UnityHttp decompress for: " + www.url);
                            }
                        }
                    }
                    else
#endif
                    {
                        Debug.Log("Payload after decompressing: " + responseText);
                        OnResponse(responseText, reqContainer);
                    }
                }
                catch (Exception e)
                {
                    OnError("Unhandled error in PlayFabUnityHttp: " + e, reqContainer);
                }
            }
            www.Dispose();
        }
apis
2 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Rick Chen avatar image Rick Chen ♦ commented ·

I will do a similar test using Android Studio. It could take a while. Your patience is appreciated.

0 Likes 0 ·
drallcom3 avatar image drallcom3 Rick Chen ♦ commented ·

Thank you. Keep in mind that I used Unity and a real APK on the device for my tests, in case Android Studio shows no issues.

0 Likes 0 ·

1 Answer

·
Rick Chen avatar image
Rick Chen answered

According to my test, when calling the LoginWithCustomID from Android emulator, the first 2 response bytes are 123 & 34. However, from the fiddler trace, I notice that the request is gzip compressed. The reason why the response bytes are not 31 & 139 is unknown. If there is any impact done by this, you could let us know.

2 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

drallcom3 avatar image drallcom3 commented ·

I managed to get Fiddler to work with the Android Studio Emulator. I also used a much simpler test:

1) GetCatalogItems() on a large catalog.

2) Check how many bytes the response body has. Gzip should compress nicely here.

Result:

  • The SDK claims the response is not compressed.
  • The SDK does not decompress the result.
  • Fiddler says it's gzip compressed.
  • Byte size clearly shows it's compressed (2500 compressed vs 35000 when forcing uncompressed).
  • Response is perfectly fine ingame.
  • SDK received an already decompressed response somehow and therefore doesn't know it was ever compressed and also doesn't need to decompress. To the SDK it looks like it was never compressed at all.

Technically a bug, since the SDK claims it's not compressed.

Realistically API compression works fine regardless and any user can ignore this bug.

At some point I'll test it on a real device, but the emulator already provides the desired scenario and result.

1 Like 1 ·
Rick Chen avatar image Rick Chen ♦ drallcom3 commented ·

Thank you for testing and providing the detailed test results. The response could be uncompressed somewhere else for this scenario.

0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.