C#•3y ago

❔ Would it be the fastest way to read text from a website with HttpWebRequest?

Im curious to know what would be the fastest way to read text from links such as https://www.gutenberg.org/cache/epub/69354/pg69354.txt There are other formats that are bigger in size but this is one the smaller ones.

var request = new HttpRequestMessage(HttpMethod.Get,"https://www.gutenberg.org/cache/epub/69354/pg69354.txt");
var response = await Client.SendAsync(request);

if (!response.IsSuccessStatusCode)
            { 
                return false;
            }

Stream stream = response.GetResponseStream();
StreamReader streamReader = new StreamReader(stream);

return streamReader.ReadToEnd(); // returns a long string

var request = new HttpRequestMessage(HttpMethod.Get,"https://www.gutenberg.org/cache/epub/69354/pg69354.txt");
var response = await Client.SendAsync(request);

if (!response.IsSuccessStatusCode)
            { 
                return false;
            }

Stream stream = response.GetResponseStream();
StreamReader streamReader = new StreamReader(stream);

return streamReader.ReadToEnd(); // returns a long string

Is there any other ways to improve it. Maybe make it thread safe probably if i'm going to make multiple url requests at once. (thats for another post).

5 Replies

mtreit•3y ago

The fastest way to read all of the data? Don't use async. Other than that you're bound by your network bandwidth speed It's I/O...you can't cheat physics There is some overhead to using async that you can eliminate but it's probably irrelevant compared to the I/O bottleneck As an aside you should dispose your StreamReader via using

HerculesOP•3y ago

Thanks for replying, what exactly do you mean with "overhead" to using async. Are you thinking of the problematic issues that can occur such as a Race Condition.

mtreit•3y ago

No, just that async isn't free...the state machine that the async plumbing uses has overhead. Not to blow my own horn but I have a number of examples in my talk on Sunday at the Solution1 conference this guild is putting on

Unknown User•3y ago

Message Not Public

Accord•3y ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ Would it be the fastest way to read text from a website with HttpWebRequest?

Did you find this page helpful?