C#•15mo ago

✅ What is IEnumerable<out T> ?

Hi! I have written this code which works, but I dont understand what type of value 'var myQuery' actually is. Is it an array containing the filtered values? // Data source. int[] numbers = new int[] { 34, 57, 89, 12, 4, 6, 7, 123 }; // Data query. <---MY QUESTION var myQuery = numbers.Where(number => number < 10); // Data Execution. foreach (var number in myQuery) { Console.WriteLine(number); }

95 Replies

Pobiega•15mo ago

no. its a lazy evaluated "sequence" meaning when you do numbers.Where(...) its not actually looping over your list and calculating anything

EmelieOP•15mo ago

Pobiega•15mo ago

its just remembering that "oh okay, I should only take the items that match this criteria"

EmelieOP•15mo ago

ok! because I found this online, and from it I understood it as being an array created, saved in the var variable. And the variable type will change according to what is retrieved.

Pobiega•15mo ago

so, thats.. uh.. not correct. and also using query syntax, which is a warcrime

EmelieOP•15mo ago

why?

Pobiega•15mo ago

do you honestly think

var myQuery = from number in numbers where number > 10;

var myQuery = from number in numbers where number > 10;

is more readable than

var myQuery = numbers.Where(number => number > 10);

var myQuery = numbers.Where(number => number > 10);

? it gets even worse when you add in even more linq methods but thats a side issue, just be aware that most people greatly prefer the method syntax

EmelieOP•15mo ago

ah you mean the version written in the example? yeah I wasnt looking at it very much, I just read the part about it being an array

Pobiega•15mo ago

okay. well, its not. its a sequence of elements, but its not an array and the important part is that its lazy

EmelieOP•15mo ago

but what is the type then?

Pobiega•15mo ago

IEnumerable

EmelieOP•15mo ago

int[] numbers = new int[] { 34, 57, 89, 12, 4, 6, 7, 123 }; // Data query. var myQuery = numbers.Where(number => number < 10); // Data Execution. foreach (var number in myQuery) { Console.WriteLine(number); }

Pobiega•15mo ago

dont care about what that is under the hood, its not important

EmelieOP•15mo ago

this is what I wrote, what exactly is wrong with it?

Pobiega•15mo ago

nothing nothing is wrong with it. but myQuery is not an array.

EmelieOP•15mo ago

ah okay I dont really understand IEnumerable, been reading about it online for a while

Pobiega•15mo ago

its a lazy sequence thats really all it is

EmelieOP•15mo ago

but if its lazy, is there still nothing wrong with the code I wrote? because lazy sounds bad

Pobiega•15mo ago

oh god no its not lazy as in bad its lazy as in "lazily evaluated"

EmelieOP•15mo ago

ok thanks! not sure if you meant lazy the good way or the bad way

Pobiega•15mo ago

😄 no its evaluated on demand let me demonstrate

EmelieOP•15mo ago

please do!

Pobiega•15mo ago

oh silly me

MODiX•15mo ago

Pobiega

REPL Result: Success

int[] numbers = new int[] { 34, 57, 89, 12, 4, 6, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

var biggestNumber = myQuery.Max();

Console.WriteLine($"The biggest number below 10 was {biggestNumber}");

int[] numbers = new int[] { 34, 57, 89, 12, 4, 6, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

var biggestNumber = myQuery.Max();

Console.WriteLine($"The biggest number below 10 was {biggestNumber}");

Console Output

Testing if 34 is below 10
Testing if 57 is below 10
Testing if 89 is below 10
Testing if 12 is below 10
Testing if 4 is below 10
Testing if 6 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
The biggest number below 10 was 7

Testing if 34 is below 10
Testing if 57 is below 10
Testing if 89 is below 10
Testing if 12 is below 10
Testing if 4 is below 10
Testing if 6 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
The biggest number below 10 was 7

Compile: 742.819ms | Execution: 152.338ms | React with ❌ to remove this embed.

Pobiega•15mo ago

so, we see that it goes through all the numbers. that makes sense. but let me demonstrate WHEN it does it

EmelieOP•15mo ago

Im all ears!

MODiX•15mo ago

Pobiega

REPL Result: Success

int[] numbers = new int[] { 34, 57, 89, 12, 4, 6, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

Console.WriteLine("MyQuery has been made, but Max has yet to be called.");

var biggestNumber = myQuery.Max();

Console.WriteLine($"The biggest number below 10 was {biggestNumber}");

int[] numbers = new int[] { 34, 57, 89, 12, 4, 6, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

Console.WriteLine("MyQuery has been made, but Max has yet to be called.");

var biggestNumber = myQuery.Max();

Console.WriteLine($"The biggest number below 10 was {biggestNumber}");

Console Output

MyQuery has been made, but Max has yet to be called.
Testing if 34 is below 10
Testing if 57 is below 10
Testing if 89 is below 10
Testing if 12 is below 10
Testing if 4 is below 10
Testing if 6 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
The biggest number below 10 was 7

MyQuery has been made, but Max has yet to be called.
Testing if 34 is below 10
Testing if 57 is below 10
Testing if 89 is below 10
Testing if 12 is below 10
Testing if 4 is below 10
Testing if 6 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
The biggest number below 10 was 7

Compile: 704.779ms | Execution: 164.711ms | React with ❌ to remove this embed.

Pobiega•15mo ago

look at that. so we create our query, print that its done, then we call myQuery.Max() to find the largest number and only then are the numbers evaluated to see if they are below 10

EmelieOP•15mo ago

you mean - nothing is actually exectuted until myQuery.Max(); is called?

Pobiega•15mo ago

yes! thats what lazy evaluation means. it runs when needed, not before

EmelieOP•15mo ago

so hence being "lazy" it wont move until it actually HAS to? 🤣

Pobiega•15mo ago

exactly

EmelieOP•15mo ago

ah thank you

Pobiega•15mo ago

however, this has a sideeffect

EmelieOP•15mo ago

which is?

Pobiega•15mo ago

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

var biggestNumber = myQuery.Max();
var smallestNumber = myQuery.Min();

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

var biggestNumber = myQuery.Max();
var smallestNumber = myQuery.Min();

what do you think the output will be here?

EmelieOP•15mo ago

probably only the smallestNumber?

MODiX•15mo ago

Pobiega

REPL Result: Success

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

var biggestNumber = myQuery.Max();
var smallestNumber = myQuery.Min();

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
});

var biggestNumber = myQuery.Max();
var smallestNumber = myQuery.Min();

Console Output

Testing if 12 is below 10
Testing if 4 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
Testing if 12 is below 10
Testing if 4 is below 10
Testing if 7 is below 10
Testing if 123 is below 10

Testing if 12 is below 10
Testing if 4 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
Testing if 12 is below 10
Testing if 4 is below 10
Testing if 7 is below 10
Testing if 123 is below 10

Compile: 801.138ms | Execution: 108.172ms | React with ❌ to remove this embed.

Pobiega•15mo ago

it evaluates the Where condition twice

EmelieOP•15mo ago

and the testing lines of course

Pobiega•15mo ago

this is what we call "multiple enumeration"

EmelieOP•15mo ago

and thats bad, I suppose?

Pobiega•15mo ago

could be what if this list had... 1000000 items? and instead of checking that its below 10, we check if some database has that item bit of a contrived example, but we obviously don't want to evaluate the condition twice if we dont need to so how do we fix that? we can "realize" the query when we know we are not going to be modifying it further

MODiX•15mo ago

Pobiega

REPL Result: Success

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
}).ToArray();

var biggestNumber = myQuery.Max();
Console.WriteLine($"Biggest number was  {biggestNumber}");
var smallestNumber = myQuery.Min();
Console.WriteLine($"Smallest number was {smallestNumber}");

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
{
    Console.WriteLine($"Testing if {number} is below 10");
    return number < 10;
}).ToArray();

var biggestNumber = myQuery.Max();
Console.WriteLine($"Biggest number was  {biggestNumber}");
var smallestNumber = myQuery.Min();
Console.WriteLine($"Smallest number was {smallestNumber}");

Console Output

Testing if 12 is below 10
Testing if 4 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
Biggest number was  7
Smallest number was 4

Testing if 12 is below 10
Testing if 4 is below 10
Testing if 7 is below 10
Testing if 123 is below 10
Biggest number was  7
Smallest number was 4

Compile: 786.596ms | Execution: 105.441ms | React with ❌ to remove this embed.

Pobiega•15mo ago

here, I added a ToArray() call at the end of the query this forces it to enumerate and stores the result as an array

EmelieOP•15mo ago

I was justing going to ask you how all this compares to an array *just what does "enumerate" actually mean? I sorta understand it but I couldn´t really explain it ... so I dont think I do lol like counting things manually, one by one?

Pobiega•15mo ago

its a bit contextual, but "looping over the source" is one way to think of it lets take Min and Max as an example to find the smallest number in a sequence, we need to check every number in the sequence right?

EmelieOP•15mo ago

yes

Pobiega•15mo ago

and the same goes for biggest, ye?

EmelieOP•15mo ago

Pobiega•15mo ago

so when I do

var biggestNumber = myQuery.Max();
var smallestNumber = myQuery.Min();

var biggestNumber = myQuery.Max();
var smallestNumber = myQuery.Min();

we are actually going over the sequence twice

EmelieOP•15mo ago

true

Pobiega•15mo ago

once for max, once for min now here is the kicker If you KNOW you only care about numbers below 10 but the list has... 1000000 items its probably better to go over the list once, filter out only values below 10, save that, then let min and max go over that result this results in a much smaller total amount of iterations than accessing the source list twice

EmelieOP•15mo ago

ah so hence why you use the array to do the first filtering?

Pobiega•15mo ago

yes! this means that its no longer an IEnumerable

Pobiega•15mo ago

Pobiega•15mo ago

you can see that Rider lets us know that myQuery is actually int[] now

EmelieOP•15mo ago

because, for some reason, when we go from an IEnumerable to array, its not as heavy to do ?

Pobiega•15mo ago

no, because that terminates the "lazy" where is lazy, remember? so when we call .ToArray(), it runs on all the items and we store the result

EmelieOP•15mo ago

ah ! because if we dont make it an array... it will loop the 1000000 items twice?

Pobiega•15mo ago

yes

EmelieOP•15mo ago

I get it I think! but the array must also loop everything 10000 etc times ?

Pobiega•15mo ago

yes, but thats unavoidable if you want every single item below 10 in a list of 10000000 items, you must check every item but its bad to do that checking every time also, methods like Where can be chained

var myQuery = numbers.Where(number =>
    {
        Console.WriteLine($"Testing if {number} is below 10");
        return number < 10;
    })
    .Where(x =>
    {
        Console.WriteLine($"Only keep {x} greater than 0");
        return x > 10;
    }).ToArray();

var myQuery = numbers.Where(number =>
    {
        Console.WriteLine($"Testing if {number} is below 10");
        return number < 10;
    })
    .Where(x =>
    {
        Console.WriteLine($"Only keep {x} greater than 0");
        return x > 10;
    }).ToArray();

MODiX•15mo ago

Pobiega

REPL Result: Success

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
    {
        Console.WriteLine($"Testing if {number} is below 10");
        return number < 10;
    })
    .Where(x =>
    {
        Console.WriteLine($"Only keep {x} greater than 0");
        return x > 10;
    }).ToArray();

int[] numbers = new int[] { 12, 4, 7, 123 };

var myQuery = numbers.Where(number =>
    {
        Console.WriteLine($"Testing if {number} is below 10");
        return number < 10;
    })
    .Where(x =>
    {
        Console.WriteLine($"Only keep {x} greater than 0");
        return x > 10;
    }).ToArray();

Console Output

Testing if 12 is below 10
Testing if 4 is below 10
Only keep 4 greater than 0
Testing if 7 is below 10
Only keep 7 greater than 0
Testing if 123 is below 10

Testing if 12 is below 10
Testing if 4 is below 10
Only keep 4 greater than 0
Testing if 7 is below 10
Only keep 7 greater than 0
Testing if 123 is below 10

Compile: 721.638ms | Execution: 156.012ms | React with ❌ to remove this embed.

Pobiega•15mo ago

look at those results. pretty interesting order if you ask me it checks 12, but 12 is not below 10 so its discarded then it checks 4, and thats kept and then we check if 4 is above 0 so it evaluates the entire chain, per item, if possible

EmelieOP•15mo ago

so if I summarize it now... if we dont force the .ToArray, then the myQuery.Max and myQuery.Min will have to work with the complete 10000+ data to filter. But, if we use ToArray, we will store all of the interesting values in an array. And only this array will be evaluated.

Pobiega•15mo ago

~~And only this array will be evaluated.~~ 🙂

EmelieOP•15mo ago

I mean, the numbers stored in the array will be evaluated?

Pobiega•15mo ago

wdym? by Min and Max? yes because they are now being called on the array, not an IEnumerable or the original source

EmelieOP•15mo ago

we do the .ToArray so that min and max can work with the <10 numbers directly

Pobiega•15mo ago

yes, at that point, "myQuery" is now an array and has no connection to the original numbers its a new array in memory

EmelieOP•15mo ago

but how is an array structurally connected to the IEnumerable? because I mean, they both deal with storing a sequence

MODiX•15mo ago

Pobiega

REPL Result: Failure

int[] numbers = new int[] { 12, 4, 7, 123, 9 };

var myQuery = numbers
    .Where(number => number < 10)
    .Where(x => x > 10)
    .Take(2);

var newArray = myQuery.ToArray();

var biggestNumber = newArray.Max();
var smallestNumber = newArray.Min();
Console.WriteLine($"Biggest number was  {biggestNumber}");
Console.WriteLine($"Smallest number was {smallestNumber}");

int[] numbers = new int[] { 12, 4, 7, 123, 9 };

var myQuery = numbers
    .Where(number => number < 10)
    .Where(x => x > 10)
    .Take(2);

var newArray = myQuery.ToArray();

var biggestNumber = newArray.Max();
var smallestNumber = newArray.Min();
Console.WriteLine($"Biggest number was  {biggestNumber}");
Console.WriteLine($"Smallest number was {smallestNumber}");

Exception: InvalidOperationException

- Sequence contains no elements

- Sequence contains no elements

Compile: 777.210ms | Execution: 111.545ms | React with ❌ to remove this embed.

EmelieOP•15mo ago

so they should be quite connected

Pobiega•15mo ago

lets rewind a bit when you make an IEnumerable like myQuery, by running for example .Where on numbers its very much still connected to the array actually, lets demonstrate that

MODiX•15mo ago

Pobiega

REPL Result: Success

var numbers = new List<int> { 100 };

var myQuery = numbers
    .Where(number => number < 10);

numbers.AddRange(new []{1, 5, 10, 20});

foreach (var i in myQuery)
{
    Console.WriteLine(i);
}

var numbers = new List<int> { 100 };

var myQuery = numbers
    .Where(number => number < 10);

numbers.AddRange(new []{1, 5, 10, 20});

foreach (var i in myQuery)
{
    Console.WriteLine(i);
}

Console Output

1
5

1
5

Compile: 656.721ms | Execution: 108.438ms | React with ❌ to remove this embed.

Pobiega•15mo ago

so we make a list with 100 in it then we create the enumerable after that we add 4 new numbers then we loop over the query and it prints 1 and 5, both numbers added AFTER the enumerable was created

EmelieOP•15mo ago

when you say "the enumerable" you mean the myQuery in this case?

Pobiega•15mo ago

yes

EmelieOP•15mo ago

but a list is not 100% like an array, or?

Pobiega•15mo ago

true. its a wrapper around an array to let it grow as needed arrays are fixed size, lists are not but thats not the important bit 🙂

EmelieOP•15mo ago

but in this example, it would work the same either with an array or list?

Pobiega•15mo ago

yes, except that arrays dont have an "AddRange" or "Add" method but if we manually added them to an oversized array, yes I just wanted to demonstrate that the datasource matters, but not the values in it at the time of the creation of the enumerable, that is 🙂

EmelieOP•15mo ago

EmelieOP•15mo ago

This part in the bottom, var myQuery = ....

Pobiega•15mo ago

yup

EmelieOP•15mo ago

is that where we create a enumerable "version" of the list ? we use the list to create a "lazy sequence"? but we can still modify the original list and the query will still work, even if we add or remove numbers?

Pobiega•15mo ago

yes

is that where we create a enumerable "version" of the list

this isnt true, but the rest is we just create an "IEnumerable" that uses the list as its source. the list is untouched

EmelieOP•15mo ago

yes, I mean the list remains as it was originally. But its just as a source to create an Ienumerable.

Pobiega•15mo ago

yes

EmelieOP•15mo ago

used as a source okay! I need to thank you very much for taking the time, this all feels a lot clearer to me now 🙂

Pobiega•15mo ago

yw its a pretty important concept in .NET so its worth the effort to understand 🙂 you can even make your own enumerables 🙂 this is a valid method:

public static IEnumerable<int> SomeCoolNumbers()
{
    yield return 7;
    yield return 7;
    yield return 6;
}

public static IEnumerable<int> SomeCoolNumbers()
{
    yield return 7;
    yield return 7;
    yield return 6;
}

it returns an enumerable that will contain 3 numbers (7,7,6). if you call First on it, it returns 7. If you call Last, 6. you can even do some very funky stuff with this

public static IEnumerable<int> SomeCoolNumbers()
{
    var i = 0;
    while (true)
    {
        yield return i++;
    }
}

public static IEnumerable<int> SomeCoolNumbers()
{
    var i = 0;
    while (true)
    {
        yield return i++;
    }
}

dont loop over this enumerable 😄 but feel free to do .Take(10) on it 🙂 and it would get the first 10 numbers

Thinker•15mo ago

Also, as a slight addendum, you can do this:

MODiX•15mo ago

Thinker

REPL Result: Success

var xs = new List<int> { 1, 2, 3 };

var odd = xs.Where(x => x % 2 == 1);

xs.Add(4);
xs.Add(5);

foreach (var x in odd) Console.WriteLine(x);

var xs = new List<int> { 1, 2, 3 };

var odd = xs.Where(x => x % 2 == 1);

xs.Add(4);
xs.Add(5);

foreach (var x in odd) Console.WriteLine(x);

Console Output

1
3
5

1
3
5

Compile: 729.959ms | Execution: 101.298ms | React with ❌ to remove this embed.

Thinker•15mo ago

This demonstrates that what odd here is actually an object which contains a reference to the list xs, because you can add new items to the list after calling Where but which will still be considered the enumerable returned by Where. What Where actually returns is something like this:

class WhereIterator<T> : IEnumerable<T>
{
    private readonly Func<T, bool> _filter;
    private readonly IEnumerable<T> _source;

    public WhereIterator(Func<T, bool> filter, IEnumerable<T> source)
    {
        _filter = filter;
        _source = source;
    }
}

class WhereIterator<T> : IEnumerable<T>
{
    private readonly Func<T, bool> _filter;
    private readonly IEnumerable<T> _source;

    public WhereIterator(Func<T, bool> filter, IEnumerable<T> source)
    {
        _filter = filter;
        _source = source;
    }
}

And Where looks like

public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Func<T, bool> filter)
{
    return new WhereIterator<T>(source, filter);
}

public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Func<T, bool> filter)
{
    return new WhereIterator<T>(source, filter);
}

Gaming

Programming

✅ What is IEnumerable<out T> ?