C#•2mo ago

Converting json to MemoryStream and back breaks the json

For context, I have the following JSON (most of the stuff omitted for brevity):

{"schemaId":"azureMonitorCommonAlertSchema","data":{...}}

{"schemaId":"azureMonitorCommonAlertSchema","data":{...}}

After I convert it to a MemoryStream and back to string, it surrounds the string with quotation marks and re-escapes the already-escaped characters. Method used to convert to MemoryStream:

internal static MemoryStream CreateMemoryStream(this string data)
{
  byte[] byteArray = Encoding.UTF8.GetBytes(data);
  MemoryStream stream = new(byteArray) { Position = 0 };
  return stream;
}

internal static MemoryStream CreateMemoryStream(this string data)
{
  byte[] byteArray = Encoding.UTF8.GetBytes(data);
  MemoryStream stream = new(byteArray) { Position = 0 };
  return stream;
}

I also tried using a StreamWriter approach, but to no avail. The snippet of code that converts the MemoryStream back to string:

 using StreamReader reader = new(ms, encoding: System.Text.Encoding.UTF8, leaveOpen: true);
 string result = await reader.ReadToEndAsync();

 using StreamReader reader = new(ms, encoding: System.Text.Encoding.UTF8, leaveOpen: true);
 string result = await reader.ReadToEndAsync();

It results in the following broken JSON:

"{\"schemaId\":\"azureMonitorCommonAlertSchema\",\"data\":{...}}"

"{\"schemaId\":\"azureMonitorCommonAlertSchema\",\"data\":{...}}"

I can't pinpoint exactly why this is happening, I guess it has to do with the fact that when it reads it back to a string, because the content is already a "string", it escapes the already-escaped quotes and surrounds it with the quotemarks. I don't know how to avoid this behaviour tho. Edit: change wording

36 Replies

Ꜳåąɐȁặⱥᴀᴬ•2mo ago

it's not memorystream it looks to me like string is being encoded one more time somewhere by json

mini•2mo ago

I'm fairly sure this might just be your IDE showing the string as escaped like that

Ꜳåąɐȁặⱥᴀᴬ•2mo ago

if you use the viewer that opens in a new window iirc it doesn't escape the content

HassOP•2mo ago

I copied the content from the viewer

mini•2mo ago

yea thats always escaped

HassOP•2mo ago

hovering the converted string:

HassOP•2mo ago

hovering the original string:

HassOP•2mo ago

the conversion surrounds the string with quotation marks and escapes again the already-escaped characters

mini•2mo ago

hmm thats weird though

mini•2mo ago

HassOP•2mo ago

agreed what if you convert it to a memory stream? unfortunately I can't work with a byte[], the method I'm calling expects a Stream whence why I convert to a MemoryStream

mini•2mo ago

I cannot replicate, can you try to output the resulting string read via the stream reader? by the way, you don't need to set Position to 0 on the memory reader, that's a given if constructed from a byte array

mini•2mo ago

HassOP•2mo ago

you mean like string result = await reader.ReadToEndAsync()?

mini•2mo ago

yes

HassOP•2mo ago

the result is the broken string

using StreamReader reader = new(request.Object.Body, encoding: System.Text.Encoding.UTF8, leaveOpen: true);
string bodyResult = await reader.ReadToEndAsync();

using StreamReader reader = new(request.Object.Body, encoding: System.Text.Encoding.UTF8, leaveOpen: true);
string bodyResult = await reader.ReadToEndAsync();

the code

mini•2mo ago

hmm in that case there is definitely some intermediate layer between the parts of code you've shown Are those two strings from different applications and transmitted via HTTP or similar?

HassOP•2mo ago

I'm testing an azure function, so the way they implemented it here is by expecting a HttpRequestData, instead of a HttpRequest

mini•2mo ago

I feel like maybe that layer causes the escaping

HassOP•2mo ago

So what I'm (trying) to do is to create a HttpRequestData and set its Body property as the json but to do so, I have to convert to a Stream, as it does not accept a string

mini•2mo ago

Are you setting the header to application/json? Maybe the receiving end expects that and otherwise escapes it or something like that

HassOP•2mo ago

I think it's plausible, but wouldn't it also break incoming jsons? I tested it using postman and it was fine

mini•2mo ago

yup I found something nvm its related to something else

HassOP•2mo ago

in the payload? lemme check I'm using a premade function

mini•2mo ago

When you manually send the request, what do you receive in postman?

mini•2mo ago

https://stackoverflow.com/questions/66911660/httprequestdata-wraps-body-in-json

Stack Overflow

HttpRequestData wraps body in { "json" : "" }

I am trying to set up an .NET 5 Azure Http function that takes a request with a json body and outputs it to the logs. using System.Threading.Tasks; using Microsoft.Azure.Functions.Worker; using Mic...

mini•2mo ago

Maybe that's of help? https://github.com/Azure/azure-functions-dotnet-worker/blob/main/src/DotNetWorker.Core/Http/HttpRequestDataExtensions.cs or this

HassOP•2mo ago

in the headers stuff? Transfer-Encoding: chunked Server: kestrel and the date

mini•2mo ago

the body

HassOP•2mo ago

what is sent? {"schemaId":"azureMonitorCommonAlertSchema","data":{...}} the same as the hardcoded string in the code I checked and I have the same packages (different version tho, as they are more recent)

mini•2mo ago

so the sending part doesn't seem to be at fault

HassOP•2mo ago

I tried expliciting the content-type for the HttpRequestData that is created, but same result

mini•2mo ago

Can you share the whole code responsible for receiving the request and outputting the resulting json?

HassOP•2mo ago

I guess I can one minute

public static Mock<HttpRequestData> HttpRequestDataMock(string data, HttpStatusCode httpStatusCode = HttpStatusCode.OK)
{
  Mock<FunctionContext> functionContextMock = new();
  HttpHeadersCollection headers = new() { { "Authorization", new List<string?> { "Bearer token" } } };
  headers.Add("content-type", "application/json");

  Mock<HttpRequestData> httpRequestDataMock = new(functionContextMock.Object);
  httpRequestDataMock.Setup(r => r.Headers).Returns(headers);
  httpRequestDataMock.Setup(r => r.Body).Returns(data.CreateMemoryStream());
  httpRequestDataMock.Setup(r => r.CreateResponse()).Returns(() =>
  {
    Mock<HttpResponseData> httpResponseData = new(functionContextMock.Object);
    httpResponseData.SetupProperty(s => s.Headers, headers);
    httpResponseData.SetupProperty(s => httpStatusCode);
    httpResponseData.SetupProperty(s => s.Body, new MemoryStream());
    return httpResponseData.Object;
  });

  return httpRequestDataMock;
}

public static Mock<HttpRequestData> HttpRequestDataMock(string data, HttpStatusCode httpStatusCode = HttpStatusCode.OK)
{
  Mock<FunctionContext> functionContextMock = new();
  HttpHeadersCollection headers = new() { { "Authorization", new List<string?> { "Bearer token" } } };
  headers.Add("content-type", "application/json");

  Mock<HttpRequestData> httpRequestDataMock = new(functionContextMock.Object);
  httpRequestDataMock.Setup(r => r.Headers).Returns(headers);
  httpRequestDataMock.Setup(r => r.Body).Returns(data.CreateMemoryStream());
  httpRequestDataMock.Setup(r => r.CreateResponse()).Returns(() =>
  {
    Mock<HttpResponseData> httpResponseData = new(functionContextMock.Object);
    httpResponseData.SetupProperty(s => s.Headers, headers);
    httpResponseData.SetupProperty(s => httpStatusCode);
    httpResponseData.SetupProperty(s => s.Body, new MemoryStream());
    return httpResponseData.Object;
  });

  return httpRequestDataMock;
}

Mock<HttpRequestData> request = HttpRequestDataMock(validPayload);
using StreamReader reader = new(request.Object.Body, encoding: System.Text.Encoding.UTF8, leaveOpen: true);
string bodyResult = await reader.ReadToEndAsync();

Mock<HttpRequestData> request = HttpRequestDataMock(validPayload);
using StreamReader reader = new(request.Object.Body, encoding: System.Text.Encoding.UTF8, leaveOpen: true);
string bodyResult = await reader.ReadToEndAsync();

The validPayload is the valid json

Gaming

Programming

Converting json to MemoryStream and back breaks the json

Did you find this page helpful?