As the Preview 4 of .Net 6 becomes available, one of the things that excites me is the System.Text.Json support for IAsyncEnumerable. The IAsyncEnumerable, introduced in .Net Core 3 and C# 8, enables us to iterate over async Enumerables. The newer version extends this support to the System.Text.Json.
Consider the following data.
[{"Id":0,"Value":"915777539"},{"Id":1,"Value":"1332243482"},{"Id":2,"Value":"306207588"},
{"Id":3,"Value":"1413388423"},{"Id":4,"Value":"2145941621"},{"Id":5,"Value":"1041779876"},
{"Id":6,"Value":"1121436961"},{"Id":7,"Value":"520045044"},{"Id":8,"Value":"1357859915"},
{"Id":9,"Value":"1340510964"},{"Id":10,"Value":"1183306988"},{"Id":11,"Value":"502467538"},
{"Id":12,"Value":"31513434"},{"Id":13,"Value":"999086707"},{"Id":14,"Value":"961728759"},
{"Id":15,"Value":"1756662810"},{"Id":16,"Value":"1018107007"},{"Id":17,"Value":"433502262"},
{"Id":18,"Value":"1784715926"},{"Id":19,"Value":"1418088822"},{"Id":20,"Value":"645106286"},
{"Id":21,"Value":"1720929044"},{"Id":22,"Value":"1102142546"},{"Id":23,"Value":"2138442183"},
{"Id":24,"Value":"208176799"},{"Id":25,"Value":"1700100438"},{"Id":26,"Value":"769308703"},
"Id":27,"Value":"1558581057"},{"Id":28,"Value":"352810944"},{"Id":29,"Value":"299925316"}]
we could now write a streaming deserialization method using the JsonSerializer.DeserializeAsyncEnumerable. For example.
public async IAsyncEnumerable<T> DeserializeStreaming<T>(string data)
{
using var memStream = new MemoryStream(Encoding.UTF8.GetBytes(data));
await foreach(var item in JsonSerializer.DeserializeAsyncEnumerable<T>(memStream))
{
yield return item;
await Task.Delay(1000);
}
}
// Data
public class Data
{
public int Id { get; set; }
public string Value { get; set; }
}
The async streams of deserialized data provides an oppurtunity to deserialize on demand, which could be great addition particularly deserializing large data.
var instance = new StreamingSerializationTest();
await foreach (var item in instance.DeserializeStreaming<Data>(dataString))
{
Console.WriteLine($"Data Item: {nameof(Data.Id)}={item.Id} , {nameof(Data.Value)}={item.Value}");
if(item.Id > 5)
{
break;
}
}
As you can observe in the output below, this would deserialiaze only on-demand.
Streaming Deserialize Demo
Inside Deserializing..
Data Item: Id=0 , Value=915777539
Inside Deserializing..
Data Item: Id=1 , Value=1332243482
Inside Deserializing..
Data Item: Id=2 , Value=306207588
Inside Deserializing..
Data Item: Id=3 , Value=1413388423
Inside Deserializing..
Data Item: Id=4 , Value=2145941621
Inside Deserializing..
Data Item: Id=5 , Value=1041779876
Inside Deserializing..
Data Item: Id=6 , Value=1121436961
As the moment, the Deseriliazation is severely limited to root level Json Arrays, but I guess that would over time as .Net 6 reaches release. Let us take a look at the serialization as well now. Turns out that is easy as well.
private async IAsyncEnumerable<Data> Generate(int maxItems)
{
var random = new Random();
for (int i = 0; i < maxItems; i++)
{
yield return new Data
{
Id = i,
Value = random.Next().ToString()
};
}
}
public async Task SerializeStream()
{
using var stream = Console.OpenStandardOutput();
var data = new { Data = Generate(30) };
await JsonSerializer.SerializeAsync(stream, data);
}
At this point, am not quite excited about streaming Serialization as I am about streaming Deserilization. This is because of the lack of usecase it might support. But am not denying there could be usecases, and over the time, I might be equally excited about it as well.
Complete sample of the code in this demo could be found in my Github
We will continue exploring the .Net 6 features in the upcoming posts as well. Until then, enjoy coding…
ByteLanguage.Net 
Streaming- deserializing JSON like this at root level is awesome. Good use case is serving large datasets of e.g. time-series stored as JSON to an external application over an API.
However this article mentions that deserializing IAsyncEnumerable to non-root level properties could be possible in the future. Unfortunately that is not possible with JSON. The implementation could never return the outer object until all of the JSON- contents of that class has been read, including the inner IAsyncEnumerable. Hence a standard List/Array- property would be just as good. It’s just a limitation of the JSON format itself. The deserializer cannot know that there are no additional standard properties after the IAsyncEnumerable- property.
LikeLike
I agree Christian, it was more of a wishful thinking.
LikeLike