Stream Throttling
Hello! I've been looking into various interpretations of stream throttling, and, while there are plenty out there I could grab and use, I wish to expand my programming knowledge. ( My apologies in advance if any terms I use are inaccurate, I'm self-taught and have an incomplete vocabulary )
How does stream throttling work, in it's most basic form? Most methods I've seen wait after a certain number of bytes have been sent through the stream, but what makes this possible? Does
Stream writing/reading run independently from other methods, such as what happens with Threads? If not, what core principal or functionality of Streams make this possible?
Thank you in advance to anyone who takes the time to enlighten me :)49 Replies
that highly depends on how u have the data available.
u could for example use
System.IO.Pipelines's Pipe, this thing has a writer and a reader.
the data producer would write into the pipe, the thing that sends to the actual socket/networkstream would use its reader to read a maximum of bytes and send it, measures how long the reading and writing took and waits for the rest
that package is also used for high performance async io stuff, so its worth looking into if u are generally interested in networkingI/O pipelines - .NET
Learn how to efficiently use I/O pipelines in .NET and avoid problems in your code.
u can do it without that as well if u have a
Stream as source (eg when u just want to send a file), but that would mostly be reinventing the wheel
this section of the linked article is basically what u want for throttling, its to configure that u dont write too much into the backing buffer while u send the data with delay
imo the correct phrase is actually "bandwidth throttling", but its pretty clear what u meant (and we r not here to bite u for ur mistakes but to help u to solve them anyway)
regarding ⤴️ u still need to keep around some kind of data structure that "slices" the time into intervals and how much bytes u have sent so far.
if u want to restrict it to lets say 1mb/sec, u dont pump out 1mb worth of data and then wait 900ms, u want smaller time slices than 1 second and the respective number of bytes u send
because usually u r on a server that has much higher bandwidth than ur client
thats how its done from the server side
if u have for example http requests, there is a Range header which the client can use to request some amount of bytes of the resource and over time adjusts how long it takes to receive that range, but thats basically the same as what i explained for the server side
and one more fundamental thing, u were talking about threads, get away from that idea. asynchronous programming is by far less resource usage.
so if u dont know much about that yet, its worth reading up on that
back in days it was common to have reader and a writer thread per connection, its also really simple to write.
but the way c# has integrated that with async methods and being able to await a task, ur code almost looks identical to the synchronous methods with the 2 threads (just that u dont have to create these threads)Asynchronous programming in C# - C#
An overview of the C# language support for asynchronous programming using async, await, Task, and Task
i hope i didnt overwhelm u, i just want to give ya some hints what to look at first as it is not a simple topic (and maybe because i had my hurdles with this topic myself)
and this most basic form is basically the fundamental form. as client u request a range of bytes and adjust for how many bytes u ask.
from the server side u can only guess or get feedback from the client "hey u can send X more bytes per tick or Y less bytes per tick"
requests with a range from the client side are as far as i know easier to deal with and is less stress for the server (the latter would have to compute it for all clients, the clients just has to compute it for themselves)
@Yui any questions still open so far?
I do have some questions, I'll post then here in a bit, I'm just extremely occupied at the moment :)
Thank you so much for the bountiful information, I greatly appreciate it
no rush, i was just nervous about if i scared ya off with all that stuff 😂
No no, I'm very happy to have received such detailed info :yuihehe:
It's exactly what I was looking for
So, what's the difference between an asynchronous operation and a
Thread, and do general asynchronous methods ( such as an async void method() ) run differently than asynchronous Tasks?
I'm still going to read that link listed, but that's always confused me a little
I ask because I've obviously utilized asynchronous Tasks before, but I've never noticed them running independently of other methods.avoid
async void methods at all costs, when these throw exceptions ur whole application can explode. the are only sort of accepted for some gui event handlers in winforms (and maybe wpf), but only because of a lack of alternative.
and regarding the difference between threads and tasks: a thread is basically a worker, a task is ... a task which can be finished by any worker
imagine the cooks in a restaurant, these are the threads, each order is a task
eg. if u cook one dish, u have a lot of time in between where u could do something else.
synchronous programming would be that u start to fry the mushrooms and stand there staring at them til they are done.
asynchronous programming would be that u r a cook and get throw the mushrooms in the pan, until they are done, do the "next" thingI saw in the link you provided that the term "thread" was used fairly frequently. Is there a difference between the term "thread" and the actual object-type
Thread?nope
its exactly the same thing
So, you have to use the object-type to start a thread on
void methods, but asynchronous Tasks are began on a thread automatically?or better to say, by "thread" u mean what is in c#/dotnet implemented as the
System.Threading.Thread class
do u know about the Action delegate type?I do, yes
Or, I've messed around with it, I don't know the specifics
thats somewhat similar
So, potentially no
in async programming u basically have a queue of tasks scheduled.
the threads basically do
(ofc its complexer than this in detail)
GetNextTaskToExecute() is a blocking method that pulls the next task to execute from some queue
now if u look at this async method:
first it will print 1. then the await happens
this is basically stopping the execution of the methodI understand the concept of blocking the execution path, I just don't know how this asynchronous behavior of async Tasks would show itself?
it remembers where it stopped. when
Task.Delay(500) (which is another task) finished, the await is triggered and enqueues the current method so that GetNextTaskToExecute() can get itSuch as with here --------^
well hard to explain. in the end, asynchronos stuff is event based. if u have to wait for something u stop execution. register a callback at what u are waiting for which would put the rest of the code u want to execute into the queue which some worker threads take to execute
maybe its better to see what the compiler really does to ur async method
u can see that here:
If it's hard to explain with words, maybe you could try writing a very simple example out?
the async code of
M() is the MoveNext()
this part is basically the console write and the await Task.Delay(500)
if (!awaiter.IsCompleted) <- because that isnt complete
the state is stored
then its registering a callback to execute the MoveNext() when awaiter completes and stops execution for now
And this allows the task to run as/on a thread without interrupting actions ahead of it, or am I being dense at the moment?
so the next time the
MoveNext() method is executed its basically at this stuff because the stored state:
(in the end u can ignore the else branch and only look at the stuff afterwards)Stephen Toub - MSFT
.NET Blog
How Async/Await Really Works in C# - .NET Blog
Async/await was added to the C# language over a decade ago and has transformed how we write scalable code for .NET. But how does it really work? In this post, we take a deep dive into its internals.
⤴️ explains it in detail
but long story short its instead of "i wait til this is done", "tell me when its done"
I'm going to go ahead and check out the article, but it sounds as if that's not a thread then, at least from my perspective? I, similar to probably many others, and accustomed to the idea that a a
Thread runs at the same time or as/parallel to the code in front of it, not afterwards.
Is that an inaccurate way of looking at it?its correct. multiple threads = simultaneously executing (maybe different) code
dotnet comes with a thread pool, whoms threads are used to execute the tasks as soon as they are scheduled
but the difference is, with async programming u could do all that stuff with just one thread
(GUI frameworks do that for example because they are not thread safe, so each update to a GUI component has to be done on the main thread)
Hmm, I see but am still not completely clicking
I wrote this very tiny piece for a console application
Through my perception of this concept, the
dothingy method should finish before the WorkAsync task.Of course, in my quick run of it, that's not the way it goes ^
if u would do
then the writelines tell the truth
ill take a quick ciggy break, i sorta have a restaurant example in mind how to explain it with the given code
Well, obviously in this case just swapping them would simply get it done, but the idea I have of asynchronous methods is one that allows the asynchronous method to execute and then step out of the way so the next lines can run while the method processes, regardless of how long it takes.
:Thumbs_Up:
reading that will probably help more tho xD
oh and $nothread
There Is No Thread
This is an essential truth of async in its purest form: There is no thread.
okay following situation:
u sit in a restaurant with another person
the waiter comes and asks for ur order
so u order a meal
now u have 2 options, either eagerly wait for ur meal to arrive (
await yourOrder;)
or to do something else, eg visiting the bath room to make place for the order because it was a 10 pounds meat pile
or to talk with the other person etc
eventually u are done taking a crap or talking, and then u want to have ur meal
in the end, no matter where the await is, u will get notified that your order is done/ur meal is served
u dont care about if the waiter cook the meal themself or if 2584238956 people were involved behind the scenes
u simply get here is ur meal and then u start doing something againI see, so as long as the Task is defined as an action beforehand, it will execute the asynchronous task before everything else?
Whereas, if you just asynchronously execute the task on it's own without defining it as an action, it waits it's turn?
its basically the difference between 1) execute something or 2) start execution somehwere else
async programming is 2)
"hey you! bring me a beer!" u dont execute this urself
u would only
await til the beer is there and maybe do something before that
(u can also just not await the beer at all and go away)Wait, so if you defined a variable resulting from the the order, such as with
It would return to that line afterwards?
Or is that a different use-case?
it would continue execution when it ca store the created meal in that
meal variable
because the await
another example, u r in a store, pushing ur cart and ask ur friend to grab something
now u have two options, either stand still idle while waiting for him (await) or go further and check out the rest of the items in the store
eventually u will want to meet up with ur friend to collect the item
nah thats another not well suited example =/It was good, I understand the actual application of it now, though I'll continue to read the sources you provided in hopes of learning more in-depth information about it.
It has clicked in my brain though, fortunately :)
imagine u have to wake up at 6am, u set ur alarm and go to sleep
while u sleep no work is done
until the alarm rings
then u start to "work" again
now imagine that what u r doing is managed by some higher being.
so while u sleep, it can control some other non-sleeping person
that higher being would be the thread
ur actions when u are not asleep would be tasks
once it has clicked it is reaaally easy 😂
Well, this was always a simple concept, which is why I was so surprised it was taking so long to click
i came from java, there wasnt anything like
async/await
there it was CompletableFuture.supplyAsync(() -> "Hello").thenApply(s -> s + "blah") or different chained calles where u simply pass delegates
writing a simple loop async was a mess
but it helped understanding how to write it asyncSo, to bring the topic of asynchronous operations back to stream throttling; they're necessary for stream throttling because you need to process the necessary information ( incoming buffer, whether or not the cooldown has been reached, etc. ) all while the stream is reading, which would be classified as asynchronous operations?
not directly, u could do it with synchronous code as well, but u would be wasting a lot of resources (threads) to achieve that
u would have basically 2 threads that are mainly in some blocking mode because the delay
mhm, makes sense