C#•14mo ago

✅ Deferred Execution or Forcing Immediate Execution

So I was reading the documentation on LINQ and I came across the two concepts i wrote in the title, now lets look at:

int[] nums = { 1, 2, 3, 4, 5};

var numQuery = from num in nums where (num % 2) != 0 select num;

foreach (int num in numQuery) { 
    Console.Write(num);
}

int[] nums = { 1, 2, 3, 4, 5};

var numQuery = from num in nums where (num % 2) != 0 select num;

foreach (int num in numQuery) { 
    Console.Write(num);
}

Now its pretty clear its deferred execution, we declare the query but its actually executed when the foreach is called. What I dont understand is why

var evenNumQuery =
    from num in numbers
    where (num % 2) == 0
    select num;

int evenNumCount = evenNumQuery.Count();

var evenNumQuery =
    from num in numbers
    where (num % 2) == 0
    select num;

int evenNumCount = evenNumQuery.Count();

Is considered Immediate execution, it still declares it and then triggers the query, in this case a foreach internally but conceptually this is pretty much the same. This though:

List<int> numQuery2 =
    (from num in numbers
     where (num % 2) == 0
     select num).ToList();

// or like this:
// numQuery3 is still an int[]

var numQuery3 =
    (from num in numbers
     where (num % 2) == 0
     select num).ToArray();

List<int> numQuery2 =
    (from num in numbers
     where (num % 2) == 0
     select num).ToList();

// or like this:
// numQuery3 is still an int[]

var numQuery3 =
    (from num in numbers
     where (num % 2) == 0
     select num).ToArray();

Calling .ToList or To.Array makes sense, you'd be executing the query in place and returning the result of the execution rather than saving the query to be executed later. Am I misinterpreting the documentation?

182 Replies

jorenOP•14mo ago

https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/linq/introduction-to-linq-queries#forcing-immediate-execution Just a heads up, I am aware that these technical details arent relevant to beginners. I've been writing C++ for a long time and just trying to understand all the C# details I read over, consider me a nerd 😄

Pobiega•14mo ago

First, I'd recommend using method aka fluent syntax instead of query syntax, makes it much easier to read and is generally what most of the community prefers. second, Count(); on an IEnumerable forces execution

jorenOP•14mo ago

Yeah, it uses foreach internally

Pobiega•14mo ago

exactly

jorenOP•14mo ago

but I dont understand why a foreach explicitly after declaring the query is deferred execution and this being any different

Pobiega•14mo ago

Oh, they're not

jorenOP•14mo ago

That's what the documentation showed me though, did I misread?

Pobiega•14mo ago

I think you might have misinterpreted it yes.

var numQuery = from num in nums where (num % 2) != 0 select num;

var numQuery = from num in nums where (num % 2) != 0 select num;

this is the lazy part.

jorenOP•14mo ago

Because the query variable itself never holds the query results, you can execute it as often as you like.

Pobiega•14mo ago

or var queyr = nums.Where(x => x%2 != 0); yeah

jorenOP•14mo ago

Wait, so its about the fact that whenever you use these methods that use forecah you arent exposed to the actual data therefore its immediate execution and its only deferred when you dont use these, and therefore are exposed to the queries data "raw" so it has nothing to do with the "declarate query -> execute query" concept, rather how the source data is handled and what is actually exposed to the programmer? that'd make sense, according to that logic this: int evenNumCount = evenNumQuery.Count(); is Immediate exection, while the first example wouldnt be.

Pobiega•14mo ago

I don't follow, and I think you are still misunderstanding the docs. let me boot up an IDE, 1 sec

jorenOP•14mo ago

take your time

var evenNumQuery =
    from num in numbers
    where (num % 2) == 0
    select num;

int evenNumCount = evenNumQuery.Count();

var evenNumQuery =
    from num in numbers
    where (num % 2) == 0
    select num;

int evenNumCount = evenNumQuery.Count();

Why is this "Force Immediate Execution" is my question I guess I get that it uses foreach internally, but lets say I wrote the same count method without using the .Count() but using a simple foreach and counting up with each entry. That would be deferred execution then no?

Pobiega•14mo ago

// not using var to explicitly show types
int[] source = new []{1,2,3,4,5,6,7,8,9,10};
IEnumerable<int> query = source.Where(x => x % 2 == 0);

// anything else at this point is defered execution.
var count = query.Count();

// immediate execution of the same query:
var r1 = source.Where(x => x % 2 == 0).ToArray();
var r2 = query.ToArray(); // same thing, but deferred. In fact, it will have been iterated twice (here and Count)

// not using var to explicitly show types
int[] source = new []{1,2,3,4,5,6,7,8,9,10};
IEnumerable<int> query = source.Where(x => x % 2 == 0);

// anything else at this point is defered execution.
var count = query.Count();

// immediate execution of the same query:
var r1 = source.Where(x => x % 2 == 0).ToArray();
var r2 = query.ToArray(); // same thing, but deferred. In fact, it will have been iterated twice (here and Count)

no, don't associate the code example under each header with what the header says what its trying to tell you is that the IEnumerable<T> (or IQueryable<T>) types always use deferred execution. They don't execute on their own

jorenOP•14mo ago

Ah okay, I see - I understand now

Pobiega•14mo ago

its only when you loop over it, or use a terminating method (Count, ToList, ToArray) etc generally, if your method takes an IEnumerable and returns an IEnumerable, it will work with the deferred execution. There are ofc ways to break this, but its "generally" true

jorenOP•14mo ago

so its only immediate when you call it at the same place you declare the query essentially

Pobiega•14mo ago

if you ever save a reference to the IEnumerable<T> created by a LINQ iterator, yes

jorenOP•14mo ago

makes sense

Pobiega•14mo ago

even if that is directly followed by an execution, your query is still there and can be executed again

jorenOP•14mo ago

yeah if you call count directly on it u cant do it again later as the results yield an int in that case

Pobiega•14mo ago

no you can, but it will be executed again ie iterated again

jorenOP•14mo ago

var r1 = source.Where(x => x % 2 == 0).Count(); I'd have to restate the query here no, I am not declaring the query anywhere here I directly call .Count?

Pobiega•14mo ago

depends look at r2 thats executing the "query" query again

jorenOP•14mo ago

yeah

Pobiega•14mo ago

even thou we already ran it when we did count = query.Count()

jorenOP•14mo ago

that uses deferred

Pobiega•14mo ago

there is no option really .Where is always deferred. But if you never save a reference to that iterator, and just consume it directly with an executor, you dont have multiple enumerations so if you need the filtered data AND the count, then its always better to actualize the query to a list/array first, then access the count on that

MODiX•14mo ago

Pobiega

REPL Result: Success

int[] source = {1,2,3};

var query = source.Where(x => x % 2 == 0).Select(x =>
{
    // lets do something stupid and add a side-effect to our enumerable
    Console.WriteLine(x);
    return x;
});

var count = query.Count();
var evenOnly = query.ToArray();

int[] source = {1,2,3};

var query = source.Where(x => x % 2 == 0).Select(x =>
{
    // lets do something stupid and add a side-effect to our enumerable
    Console.WriteLine(x);
    return x;
});

var count = query.Count();
var evenOnly = query.ToArray();

Console Output

2
2

2
2

Compile: 515.921ms | Execution: 107.421ms | React with ❌ to remove this embed.

Pobiega•14mo ago

oh, should have placed the select before the where. silly me but whatever, you get the point the console output we see here is from the iteration taking place in count, and then the iteration taking place in toArray

jorenOP•14mo ago

wait when using the Method Syntax you open with select rather than having it at the end I recall seeing its being the other way around with the non syntax way of defining queries

Pobiega•14mo ago

oh, dont confuse the Selectwith the query syntax select its.. different Select() is Map() from other languages its an A -> B transformation

jorenOP•14mo ago

std::map<T, Y>?

Pobiega•14mo ago

probably. I don't do cpp

jorenOP•14mo ago

what does Select return

Pobiega•14mo ago

Select<T,TResult>() returns TResult

jorenOP•14mo ago

{2, 3 },
{6, 4 },
...

{2, 3 },
{6, 4 },
...

Ill check its definition sec

Pobiega•14mo ago

Pobiega•14mo ago

here you go 🙂 right now its int,int since thats what I gave it the select here was only to trigger the console writeline its not neede

jorenOP•14mo ago

ok so it takes the Source Type and the Result type

Pobiega•14mo ago

int[] source = {1,2,3};

var query = source.Where(x => x % 2 == 0);

var count = query.Count();
var evenOnly = query.ToArray();

int[] source = {1,2,3};

var query = source.Where(x => x % 2 == 0);

var count = query.Count();
var evenOnly = query.ToArray();

this is equally fine, but no output

jorenOP•14mo ago

and does smth, and then TResult encapusulates, what exactly?

MODiX•14mo ago

Pobiega

REPL Result: Success

int[] source = {1,2,3};

var query = source.Select(x => $"{x} is a number.");

Console.WriteLine(string.Join("\n", query));

int[] source = {1,2,3};

var query = source.Select(x => $"{x} is a number.");

Console.WriteLine(string.Join("\n", query));

Console Output

1 is a number.
2 is a number.
3 is a number.

1 is a number.
2 is a number.
3 is a number.

Compile: 569.273ms | Execution: 103.021ms | React with ❌ to remove this embed.

Pobiega•14mo ago

does this make it clearer?

jorenOP•14mo ago

Join surely is an amazing method lol it can take a query like that damn

Pobiega•14mo ago

my select function now is int,string not really it takes IEnumerable<T>

jorenOP•14mo ago

yes which queries derive from no? or IQueryable i guess

Pobiega•14mo ago

sure but its not relevant really IEnumerable is just the concept of "you can use foreach on me" thats literally all it is

jorenOP•14mo ago

ah ye so he x is Source and the result that the operation yields is what it returns

Pobiega•14mo ago

yes

jorenOP•14mo ago

so TResult is whatever behind the => yields

Pobiega•14mo ago

C# uses a lot of type inference, so it understands that its int, string here yes

jorenOP•14mo ago

type deduction is what I call that return type deduction

Pobiega•14mo ago

type inference is the official term

jorenOP•14mo ago

I'll have to change my terminology anyway, it differs quite a bit from C++

Pobiega•14mo ago

https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/generics/generic-methods

Generic Methods - C# Programming Guide - C#

Learn about methods declared with type parameters, known as generic methods. See code examples and view additional available resources.

jorenOP•14mo ago

Yeah I read that one, though im not really fond of generic types in C#

Pobiega•14mo ago

var query = source.Select<int, string>(x => $"{x} is a number.");

var query = source.Select<int, string>(x => $"{x} is a number.");

jorenOP•14mo ago

its hella limited i have issues implenting generics according to what im used to

Pobiega•14mo ago

coming from templates in C++, sure :p but they are actually really powerful

jorenOP•14mo ago

I end up having to rethink my whole approach yes and much safer

Pobiega•14mo ago

yeah less footguns in general

jorenOP•14mo ago

yeah and better compile time errors 😓 Okay, so I understand Select() now though

Pobiega•14mo ago

great. Where is just Filter and SelectMany is Bind

jorenOP•14mo ago

seems easy enough, can I add constraints to Select

Pobiega•14mo ago

wdym?

jorenOP•14mo ago

or do I just cast inside Select if I need a specific type as TResult

Pobiega•14mo ago

you can do whatever you want for Tresult you provide the method that maps TSource to TResult if you want to limit the TSources that might go in, you do that with some filtering method before the select

jorenOP•14mo ago

how does one go about that Where()?

Pobiega•14mo ago

theres some options I assume we're talking about you having a list of several concrete types but all sharing some base class/interface?

jorenOP•14mo ago

yes Like IsDerived from or smth alike

Pobiega•14mo ago

OfType<T>() will filter on the concrete type if you need more control, Where can also be used where you just need to return true or false based on the actual item

jorenOP•14mo ago

yeah and filter it there makes sense and =>, this piece of syntax sugar it always yields to a standalone method? or is it a lambda under the hood or does it differ on the context in which its used In Select()'s case, a lambda would make sense however in a class like:

class A
{
  int smth() => 2 + 2;
}

class A
{
  int smth() => 2 + 2;
}

MODiX•14mo ago

Pobiega

REPL Result: Success

var source = new List<Base>
{
    new Base(),
    new Mid(),
    new Top()
};

var query = source.OfType<Mid>();
foreach (var x in query)
{
    Console.WriteLine(x.GetType().FullName);
}

public class Base { }

public class Mid : Base { }

public class Top : Mid { }

var source = new List<Base>
{
    new Base(),
    new Mid(),
    new Top()
};

var query = source.OfType<Mid>();
foreach (var x in query)
{
    Console.WriteLine(x.GetType().FullName);
}

public class Base { }

public class Mid : Base { }

public class Top : Mid { }

Console Output

Submission#0+Mid
Submission#0+Top

Submission#0+Mid
Submission#0+Top

Compile: 700.653ms | Execution: 109.636ms | React with ❌ to remove this embed.

Pobiega•14mo ago

lambda

jorenOP•14mo ago

damn, List has that method built in

Pobiega•14mo ago

no it doesnt :p

jorenOP•14mo ago

extension method?

Pobiega•14mo ago

yes same as for array above

jorenOP•14mo ago

fsss i hate those

Pobiega•14mo ago

why?

jorenOP•14mo ago

I'd rather derive from a class, implement it ontop of it

Pobiega•14mo ago

LINQ is almost entirely based on them

jorenOP•14mo ago

and call it a day

Pobiega•14mo ago

lol

jorenOP•14mo ago

ye I guess I need to adapt

Pobiega•14mo ago

extension methods are awesome because what if the class is sealed? or you otherwise can't inherit from it like, it'd be kinda shitty to have to derive every single collection just to add LINQ support what if a library you want to use wasnt updated and used non-linq types

jorenOP•14mo ago

I mean if they all should be able ot handle OfType just put it in the base clase mark it virtual and ensure its overrided though that'd give some overhead I suppose or have a base impl that suffices for all its derived types I am not used to using constraints like this either way though in C++ its not connected to a class to begin with you just call it independtly and give it both types

cap5lut•14mo ago

thats more a SRP thingy

jorenOP•14mo ago

SRP?

Pobiega•14mo ago

Or just have a static method that pretends to be part of the object? That's what extensions are

cap5lut•14mo ago

single responsibility principle

jorenOP•14mo ago

ah, didnt know that was the abbrevation Fair, ill embrace it I promise just need to get used to it I guess

cap5lut•14mo ago

extension methods can be really powerful yeah

jorenOP•14mo ago

yeah that way you dont always have to rewrite containers if they lack certain things

Pobiega•14mo ago

hardest part of learning a new language isnt learning the new language, its unlearning the habits from your old language 😄

jorenOP•14mo ago

Too bad I work in the C++ industry and C# soon

Pobiega•14mo ago

the first time I coded javascript or rust was not pretty

jorenOP•14mo ago

so I need both

Pobiega•14mo ago

thats fine you dont need to "replace" them, just learn to not force them ie, dont try to force C# to be C++ and the other way around

jorenOP•14mo ago

its hard when you dove so deep, but im trying haha

cap5lut•14mo ago

i came from the java world so i didnt have to unlearn too much ;p

jorenOP•14mo ago

C# is such a relaxing language comparabily I like it its not as verboose and complicated all the time, its refreshing Uni made you? Because mine sure as hell forced us to use Java... @Pobiega tysm for your help I'm sure ill bother you again in the future ;)

cap5lut•14mo ago

nah, started out with pascal/delphi back in school, then touched a bit of php and then was looking for some platform independent language for some CLI tools, so ended up with java

jorenOP•14mo ago

thats tough, guess there werent as mny viable options back then and java was more popular a decade ago, than it is now

cap5lut•14mo ago

yeah that was around 2006-7 iirc the php -> java transition was also because of a chatbot for a mmorpg i was playing back then which was written in php and i was too sick of duck typing

Pobiega•14mo ago

yw just make a thread here like you did now 🙂

jorenOP•14mo ago

👍

Pobiega•14mo ago

and when you are happy with your answers and want to mark a thread as done, /close it

cap5lut•14mo ago

uh sorry for the thread drift 😒

jorenOP•14mo ago

Back then I didnt think about anything programming related I was a damn child

Pobiega•14mo ago

😅 I started coding in 94..

jorenOP•14mo ago

Thats crazy, you two are old and wise huh

Pobiega•14mo ago

my wife would agree with the "old" part

jorenOP•14mo ago

'94 so you write C, ASM?

Pobiega•14mo ago

lol nah

jorenOP•14mo ago

I dont even know what was the language to write

Pobiega•14mo ago

vb6 at first, then PHP, then C# (in 2002)

cap5lut•14mo ago

i started 2000, my older siblings had it delphi in school and i wanted them to teach me

jorenOP•14mo ago

simpler times for programming I assume?

Pobiega•14mo ago

in a way

jorenOP•14mo ago

guess less resources but less options

Pobiega•14mo ago

yeah, exactly

cap5lut•14mo ago

for me it was harder times, living in the sticks, no internet and the local libraries resources were sparse

Pobiega•14mo ago

no youtube videos that went out of date in half a year after being released...

cap5lut•14mo ago

i still dont like yt tutorials

Pobiega•14mo ago

but it was also slower progress than now. A talented and motivated person can become "fluent" so damn fast these days, given the right resources yeah same

jorenOP•14mo ago

I find programming, in a way very tiring nowadays, if I dont keep practicing I lose the ability to write TMP in C++ for instance. So many details, different libs/software/hardware stacked upon eachother I touched some web dev recently and had to learn 100 different things

Pobiega•14mo ago

oh yeah for sure, if I stopped for a year or so it would be hell to come back into it and C# is fairly stable, tbh javascript, you'd have to relearn the entire ecosystem

jorenOP•14mo ago

django, gunicorn, nginx, postgresql, docker, kubernetes, etc it just keeps stacking that's good, same with C++ they just add new things but not as fast as compiler support is slow they add more than the compiler devs can handle, so its all slowed down

Pobiega•14mo ago

web is massive, especially if you include hosting and database like you did there :p

jorenOP•14mo ago

but the complexcity is disgusting, too many things to take into account good design is crucial gets tiring yep, and we didnt even mention the front end which is framework ontop of framework unless you want to reinvent the wheel and spend 3 years building a frontend

Pobiega•14mo ago

yeah web is rough. I do it for work thou 🙂 but only backend.. and database... and devops..

jorenOP•14mo ago

and its paid much less than some desktop orientated jobs though

Pobiega•14mo ago

eh, not really for a junior FE, sure

jorenOP•14mo ago

like, my entry salary with a C++ was like 100k

Pobiega•14mo ago

its all supply and demand all the bootcamps and schools are churning out javascript developers they are dime a dozen

jorenOP•14mo ago

True, and web dev is filled with people trying to learn it

Pobiega•14mo ago

well thats where most of the demand is so it makes sense, in a way

jorenOP•14mo ago

that'd balance it though and make the salaries better

Pobiega•14mo ago

yesn't I have no idea how accurate this is, but imagine that 80% of dev work today is web but 90-95% of people who are learning learn for web so the pool of people looking for entry level web jobs just keeps getting larger and larger

jorenOP•14mo ago

then the demand is low in comparison to its supply

Pobiega•14mo ago

meanwhile, the people who target a lesser used tech, like c++ desktop, sure they have slihtly less jobs to look for, but there are a lot less competition per job yeah exactly

jorenOP•14mo ago

its relative is what you're trying to say I guess 😄 so why'd you pick web dev though

Pobiega•14mo ago

so while web has the highest demand in raw numbers, the relative demand is lower because I picked 20 years ago when the numbers were different

jorenOP•14mo ago

how old are you now

Pobiega•14mo ago

jorenOP•14mo ago

not that old then

Pobiega•14mo ago

I started with PHP at around.. 14? 12?

jorenOP•14mo ago

bit earlier than me, good age to start what made you stick around though

Pobiega•14mo ago

It was fun

jorenOP•14mo ago

because I find programming, fun, but it has its

Pobiega•14mo ago

no other reason

jorenOP•14mo ago

well less enjoyable parts

Pobiega•14mo ago

I never imagined it as a career

jorenOP•14mo ago

no?

Pobiega•14mo ago

not until 17-18 like I said, this was back in the 90s

jorenOP•14mo ago

do you still program as a job, or did you take a manager role later into your career?

Pobiega•14mo ago

senior dev/devops engineer

jorenOP•14mo ago

mhm, so you're still fully into it basically

Pobiega•14mo ago

more or less not a managerial position at least

jorenOP•14mo ago

and you prefer it like this?

Pobiega•14mo ago

I've been a lead dev and architect before, but enjoyed those roles less in a way

jorenOP•14mo ago

makes sense well they can be more burderning I assume

Pobiega•14mo ago

I'm not a people person, but I'm very good with tech so I try to avoid the peter principle

jorenOP•14mo ago

makes sense peter principle is to move up the ladder at some point to managing positions Never heard of it

Pobiega•14mo ago

promotion to your degree of incompetence

jorenOP•14mo ago

Pobiega•14mo ago

ie, "you are a good dev, become a lead dev!" "you are a good lead dev, become a manager" then you suck as a manager, because they need people skills, not tech skills

jorenOP•14mo ago

yeah some pretty obvious example though I suppose skills can be learned even those

Pobiega•14mo ago

for sure

jorenOP•14mo ago

but if you're not happy with it then you shouldnt adapt also the process of adapting can be pretty rough and inefficient

Pobiega•14mo ago

I'm swedish, and here a mid-level manager can easily make less money than the devs they manage I get paid well, so I dont feel like I need to jump to manager for the salaray

jorenOP•14mo ago

thats interesting, that makes sense

Pobiega•14mo ago

its probably a local maximum scenario thou

jorenOP•14mo ago

for me its kind of the same though mostly because managers in general get paid less than C++ engineers

Pobiega•14mo ago

going via manager to a C level position would definately be a huge pay raise, but its also changing field entirely and my current company is a startup, so we dont have or need a dedicated CTO

jorenOP•14mo ago

yeah but you wouldnt be able to do that no? jump to C level you'd have to start from the bottom or somewhere in the middle and work your way up progressively over the years

Pobiega•14mo ago

yeah thats what I mean, go manager -> mid level -> top level -> CTO

jorenOP•14mo ago

I mean unless you turn out to be a natural talent surely you wont be put in the highest paying level

Pobiega•14mo ago

exactly its a huge risk and if I decided nah, lets go back to dev, im now out of shape and my CV shows managerial positions so might be hard to go back alr, gotta go back to working. gl hf in C# land

jorenOP•14mo ago

thanks man, gl with work!

Gaming

Programming

✅ Deferred Execution or Forcing Immediate Execution