petergrimsehl avatar image
petergrimsehl asked

CloudScript (Legacy): http GET request returns invalid file data


I'm using a legacy cloud script where I access a file stored in the title player player account by making a GET request to the DownloadUrl provided in the meta data retrieved by GetFiles(). The file contains binary data that was uploaded by the client using the File / Data REST API and libcurl to put the contents. This file should be uploaded to another server to be attached to an open issue.

I verified that the uploaded file contents are valid by manually downloading the file from the players file section.

Here is a part of the script up to the GET request.

 // ...
 var entityProfile = context.currentEntity;
 var getFileRequest = {
     Entity : entityProfile.Entity
 var file_url;
 var result = entity.GetFiles( getFileRequest );
 if ( result && result.Metadata )
     var file_meta = result.Metadata["bug.t7s"];
     if ( file_meta )
         file_url = file_meta.DownloadUrl;
     if ( file_url )
         var headers = {
             "Accept": "application/octet-stream"
         var get_response = http.request( file_url, "get", "", "", headers, true );    
 // ...

After executing the this GET request, get_response.length contains the correct number of characters (== file size), but debug printing the individual characters of the string get_response showed that several characters of the original byte-sequence where replaced by the unicode replacement character (0xFFFD), which renders the received file contents invalid for further processing.

The final step would be a REST call to an external service, using get_response as content of a multipart/form-data post request

 // Create the attachment.
 var auth = "omitted auth string";
 var headers = {
     "Authorization": auth,
     "X-Atlassian-Token": "no-check"
 content =
 "--partboundary" +
 "\r\nContent-Disposition: form-data; name=\"file\"; filename=\"bug.t7s\"" +
 "\r\nContent-Type: application/octet-stream" +
 "\r\n" +
 "\r\n" + get_response +
 var url = "omitted target url";
 var httpMethod = "post";
 var contentType = "multipart/form-data; boundary=partboundary";
 var response = http.request(url, httpMethod, content, contentType, headers, true);

This calls succeeds but posts invalid / broken data to the target site.

Have you any advice how I can solve this problem?

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Neils Shi avatar image
Neils Shi answered

Could you tell us why you want to get the entity files via CloudScript? Since some characters will be replaced by Unicode replacement characters in CloudScript, you may call the API GetFiles on client side. And the CloudScript is a legacy feature, we suggest you use the CloudScript using Azure Functions.

1 comment
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

petergrimsehl avatar image petergrimsehl commented ·

The clients should not be able to upload data to the external service.

But since it seems that the CloudScript API is no longer maintained, this issue can be closed.

0 Likes 0 ·
Neils Shi avatar image
Neils Shi answered

When you convert binary data to string in JavaScript, the Unicode replacement character (0xFFFD) is used to replace an incoming character whose value is unknown or ill-formed in a Unicode Transformation Format (UTF) conversion. Could you tell us more details about how you debug print the individual characters of the string “get_response?

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

petergrimsehl avatar image petergrimsehl commented ·


I'm using the follwing function for debug output

     function stringToBytes ( str ) {
     "Converting string of length: " + str.length );
         var ch, st, re = [];
         for (var i = 0; i < str.length; i++ ) {
           ch = str.charCodeAt(i);  // get char 
           st = [];                 // set up "stack"
           do {
             st.push( ( ch & 0xFF ).toString(16).padStart(2, "0").toUpperCase() );  // push byte to stack
             ch = ch >> 8;          // shift value down by 1 byte
           while ( ch );
           // "Stack has number of elements: " + st.length );
           // add stack contents to result
           // done because chars have "wrong" endianness
           re = re.concat( st.reverse() );
         // return an array of bytes "Returning array of length: " + re.length );
         return re;
0 Likes 0 ·
Neils Shi avatar image Neils Shi petergrimsehl commented ·

I cannot reproduce your issue. After I uploaded the binary data, I can get the correct response without unicode replacement character (0xFFFD), you may share us with your uploaded binary data so that we can do some research.

0 Likes 0 ·
petergrimsehl avatar image petergrimsehl Neils Shi commented ·

I uploaded a test file containing 256 bytes (0-255 ascending).

My debug print function outputs an array of 512 bytes for get_response after the GET call:

 (... for the rest of the file, ommited due to character limit in replies.)

So apparently all byte values >= 0x80 are broken.

Downloading directly from the players file section gives a file with the correct contents.

Downloading the file as attachment after uploading it to the external server with multipart/formdata gives a file of size 512 bytes with broken data after byte value 0x7F, mostly replaced with 0xBDEF.

0 Likes 0 ·
Show more comments

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.