Trying to map values to a single DTO
I have three separate tables, one for the videos, one for all possible genres and one that holds the videoID and genreID so I can tell which video has which genre(s).
Currently it makes sense that I'm getting all possible genres populated instead of only the ones that are actually for the videoID.
How can I solve this properly?
GenreRepository.cs
:
213 Replies
VideoRepository.cs
:
Genres Table
:Video Genres Table
:one that holds the videoID and genreID so I can tell which video has which genre(s).You don't need this table. You can achieve this using Relations
I understand, but it was set up in a way that I shouldn't alter for now as I need to work with it as it is table-wise.
you can create a foreign key in the
Videos
table i.e GenreId
and a navigation property Genre
then you can access the Genre of the videoAnother important piece of code may be the following:
see this is ineffecient, you're fetching all genres and then finding the one...instead you can have a ForeignKey through which you can get the specific genre of the Video in constant time which would be much faster
Bit more context to my DB relations:
Yeah, and it's not even going to work properly as I'll be filling the list with a different kind of value if it isn't a match
so you can simply alter the table, set the relations and then it'd make your life much easier
But suppose I couldn't? What approach would make it work?
you can, why you couldn't ?
Or is the current approach simply broken and can not even be referenced properly?
current approach is inefficient and not recommended at all
I'm basically working on a study project that got set up for me (not school work, don't worry). But since the idea is to learn about .NET and C#, the databases got set up for me.
And a little basic code layer was set, from there I am expanding
I'm under the impression that there is already a reference inside the tables towards the ID of both video and genre
then
that's it
No wonder why I come across stubborn
since it's a study project so you need to look for the recommended and efficient approaches to solve different problems
So inside my videos table I add a column for a genreId?
right
That column is then referencing the genre table? and sets the reference on the row with the matching Id?
right
and then you can use
JOIN
to only select those videos that have matching genre ids in the genre table
or you can use Left Join
to select all the videos no matter they have genre or not
and you can use Right Join
to select all genres , no matter they've videos or notI'm kind of scratching my head now, because now comes the second question, and maybe it's no issue at all, but alas:
What if I want multiple genres for a single video?
And something makes me think that that's the reason the tables have been set up in that fashion.
So in this case the video_genre table references two identical videoId's but a different genreId, then both of those genres get mapped into the DTO as a List
yes in case you wanna store multiple genres, you would need a separate table because it'd be a many to many relationship then
correct
I still appreciate what you just told me though, wasn't wasted!
But now I'm still stuck at step 1
You can use a three-way join
Query the video, then join all genreids that match, then join the genres
for an m:n relation neither the videos nor the genres table would have FKs.
only a a third thable that links these together (a genreID, a videoID, where both are FKs to the respective tables' PKs and its own PK would be over these two columns)
for querying that how u want there are different JOIN variants, depending on ur needs
can you show your
VideoDto
you would need a list for genres in that class , and can use Joins to fetch all the genres related to that video. For that you would need to modify the query in your GetAll
function of videos repository
I'm reading the JOIN stuff right now at https://learn.microsoft.com/en-us/sql/relational-databases/performance/joins?view=sql-server-ver16
But any extra help meanwhile is appreciated
try this maybe ?
JOINS can also be bad instead of doing multiple queries.
imagine a video that has a lot of genres, u would basically receive something like the following for JOINS
so u have a lot of redundant data in there
you can only select the columns you need and skip the others for each table
How would I alter the
GetAll
method?just replace the query and debug how the data in result looks like
.
and then adjust the mapping to dto according to the structure of data being received
"SELECT [id], [title], [duration],[image_url],[description] [genre.name]\r\nFROM [dbo].[videos] \r\nINNER JOIN [video_genres] ON [genres.id] = vg.video_id\r\nINNER JOIN genre ON vg.genre_id = videos.id;"
Is what I have till now, do I not need those brackets at all, and how do I finish that last part?
Or is it [genres].[id]?
you would need to adjust the tables according to your current setup, I've shown it as an example
rename to match to your setup
"SELECT id, title, duration,image_url,description genre.name\r\nFROM videos \r\nINNER JOIN video_genres ON genres.id = vg.video_id\r\nINNER JOIN genre ON vg.genre_id = videos.id;"
Gives me an error, so I assume I need to change it indeed
When do I need to use the square brackets surrounding the key words?
Basically coming back to this: Or is it [genres].[id]?
in normal sql i.e in dbms tools I write sql like that but perhaps in c# you need those brackets etc, I haven't use raw sql in c# much
you can try putting brackets and see if it works
Just getting exceptions
uh
pls make sure table names match the tables in your actual database
and columns as well
oh
there's a mistake in the query , my bad
I did try to fix a typo and match the actual table and columns
it should be perhaps
instead of
vg
it should be the name of that third table that's storing the reference to both videos and genres i.e video_genres
or whatever the name you've setyou need brackets when your names conflict with keywords
I see
Cause I tried the following:
"SELECT [id], [title], [duration],[image_url],[description] [genre].[name] FROM [dbo].[videos] Inner Join [dbo].[video_genres] ON [dbo].[genres].[id] = [dbo].[video_genres].[video_id]\r\nInner Join genre ON [video_genres].[genre_id] = [dbo].[videos].[id];"
No success, same exception as above.
what exception is that?
While "SELECT [id],[title],[duration],[image_url],[description] FROM [dbo].[videos]" actually retrieves all genres (but not exactly what I want).
SqlException: Incorrect syntax near '.'.
hmm
I have a repo on github, if that helps?
Maybe I'm not giving enough information
your join doesnt make sense
look at that first join
you are querying
dbo.videos
, but your join doesnt refer to it
you cant join on an unrelated tableThen I have no clue how I actually can solve this
Cause I'm unfamiliar with SQL
I am more familiar with C# than SQL, and I'm not even a beginner IMO.
?
select from v, add vg, add g
Same exception
try this
ugh, dont wanna spin up a SQL server 😄
Again, same 😛
damn
No worries, I'm frustrated too
wait a min
show me your tables in the database
still the same error about the
.
?
step 1: do the query in SSMS first
get it to work there, before doing it from C#yeah that's better
cause it seems you have a syntax error somewhere, not a logic error
are you SURE its this query?
cause I see no syntax errors
Gotcha
ah yeah
no comma between description and
genre.name
right, so fix the ambigiousness
you'll need to prefix the columns with their table alias
v
not videos
you have aliased the tablewhen using aliases you would need the aliased name in the queries
same thing applies to
g
tried that
last row
your table is called
genres
not genre
you certainly dont need that much brackets btw :p
[name]
, yes
the others... noAlmost there
invalid comma after
videos v
It's working
But now I see the redundancy
It's getting the value twice?
yeah
because you have two genres, you get two joined rows
And I still need to fix the part where it now populates the genres that belong to that video id only
Now it's still behaving as previously, I suspect because I haven't touched the return DTO yet
here is a visualization of all SQL joins
Thanks 🙂
so whats happening is that it starts on video, joins in the
vg
.. but if there is more than one hit, you now have 2 rows
and then it joins those rows on g
result: 2 mr beansYeah but now I altered the table so that mr bean has 2 of the same genres
So I want to see Family 2x
But it's still showing all genres due to the code I just sent (I think)
well yeah
How would I map the data to only retreive the matching genres?
take a step back
ignore your C# code for now
Alright
make this work in SSMS with a single query
because this is doable in ONE query
Yeah but the issue is that I have 0 SQL experience
I only select top 1000 for work to check data 😛
I'm more confused now than I started off with
I assumed it would have been more of C#
well
you can fix this with C# and running multiple queries
What's the C# way?
like, fetching all the genres, then fetching all videos and joining their genre associations as keys and then in your mapping looking up the actual names
but thats... really bad code
Cause I finally managed to get data from different tables into a single DTO, that felt massive to me.
And yeah, that sounds a bit redundant
The project was set up for me with the 3 tables
And I have no effing clue what to do as I felt like I needed to leave them alone and get it working from the C# (majority).
Might need a little break 😛
nah dont worry
its easy enough to fix
we just need to introduce a bit more stuff to your query, namely
GROUP BY
Will you still be here in like 15-20 min?
Will have a quick cup of tea and cool-down to try again :p
ta da
replace table/column names as needed
😄
😮
im sure some SQL genie can fix this even better
but it works
Does my solution seem okay-ish as well?
Cause it works..
your code has a huge problem, if you ask me
and thats that it uses 2 queries
Please share
ie, you first poll the video, then poll the genres
Yeah query -ception?
with two joins and a group, you can do all that work in one query
the database is a lot better than we are at fetching data, so doing 1 query instead of 1+n
actually, I was understating the problem before
its not 2
its 1+n
where
n
is the number of hits from the first query
5 videos? 1+5
100000000 videos? 1+100000000I agree with you
It's just that I understand C# slightly better than SQL :p
But I will commit both solutions for references
And in the mean time figure out both step by step
heard of/worked with LINQ?
So I can actually understand why it finaly works
Heard of yes, worked, probably not
right
its pure C#, but uses the same concepts, even called the same thing
I think I'll need your help again transforming your query to my tables and the aliases
how so?
Wait
This might be it
But the group by m.title
Why is that part needed?
well, we end the statement by grouping
by the nature of groups, we can only use a function over the grouped values, or a value that was part of the group key
STRING_AGG
joins all the genres in each group together
m.Title
was added to our key, since we already know it will be the same for each id
there is no field called
id
Do you mean the reader ?
Or that it's
Id
and not id
?read the exception name
and value
Yeah I'm lost
how my dude, how
what is an index
I know what an index is, don't worry.
what is indexing something
look at the parameter in the exception
id
And it's indexing something that's literally not within the same allocated memory block
its lowercased
correctamundo
again, look at your SQL
so we have a field called
Title
, an unnamed field, and... thats it
no field called id
I had no clue it was an issue from the query
So we're not grouping any Id
we are not SELECTING any id
only the things you select are available in the result
and C# can only interact with the result
This is going to be a long day
nah man, my summer vacation started 2 hours ago
we gucci
Lmfao
You deserved your vacation
Long story short; this SQL stuff isn't getting me anywhere as I have never read up the basics of it. Feels like diving deep into Node.js libraries without even understanding JS.
well, I can highly recommend https://sqlbolt.com/
its an interactive step-by-step sql tutorial
very good
and free
I'll book mark it, but for now I'd like to be spoon-fed a solution 😛
I can't learn too much at once, gets a bit demotivating if one relies on the other and I'm crap at both.
Ive already fetched the spoon, dipped it in the baby food and its now hovering outside your mouth
you will need to learn forward and open your mouth
Usually the plane flies right in
ignore sqlbolt for now thou, thats fine 🙂
the plane is hovering my guy
you just need to do that tiny last bit of effort
Hahaha, alright let me have another look, give me a bit though
and select the ID, and give a name to our genre
i'll give you a hint
Do you think there is perhaps a correlation between what we select and what shows up in the results?
Wait till Easter comes round
And yes, I do think there is a correlation ha
cool
so, what do we add/change to/in the sql query to also include the movie title, and be able to access the genres from C#?
Ah wait
your SQL is first of all is grabbing them by separating by comma's, right?
So my STRING_AGG needs changing anyhow
the genres? yes
That would differ to yours, right
only by column names
Yeah, I'm the R-word right now
First thought that was your table as-is.
Not that it was the already executed query we're discussing now
these are my tables
as is
Thanks, that clears it up
Just the table bit, so far.
the names are stupid, I know
I copypasted from some random webpage
I'm in no position to assign stupid-ness to anyone right now 😄
Step by step:
1. We are selecting m.Title column from videos table (right)?
thats really not step 1
but.. yes
I'm changing around stuff hitting in the dark now.
id > video_id
Why the hell is it out of range?
how do we fix
STRING_AGG
being readable from C#?I have no clue how STRING_AGG even works in depth
Or even surface level
Besides that it seems to concatenate in this case
you dont need to
Concatenating the selected results into a comma separated string
Based on your separator arg
what it does is join several results into one
how it does it, you dont care about
what you MUST do, is give it a column name in your result set
it currently has no name
How does one set a column name in SQL
I mean in the query
look at this image closely
Yeah I do not have the
as
value partright
so... what do you think
as
does?sets the title to the value
of the column
Sorry.
it gives the column a name, yes
Sets the column header for our title
yes!
so, we are fine with
Title
as a name for m.Title
so we dont need to give it a new one
but the STRING_AGG
one needs a name...
can you show me the updated SQL code?Bit scared to now 😛
yes!
okay, so now our result contains 2 things
title
and testing
but we also want the video id, right?Yes
right
Executing this results in an exception still, by the way
Not sure if that's expected right now :p
I dont care.
because your C# code is busted
Well my issue is
It's been executing in C#
Instead of testing in MSSMS
well thats just silly.
Cause this would have made me feel a lot better
Lol
Im pretty sure I told you to ignore C# for now and get it working in SQL first :p
I blame myself, don't worry
Yeah, 8 hours of work already done
okay so
look at the results tab down there
only 2 things
we need a third
what do we do?
We add an extra row with the as value (for the header) on the right side, the left side becomes the column value
you lost me
wat
I mean, you can make it
dont think its what you want thou
you don't need the
as
there
you only need as when you are selecting function results
anything from an actual data column will default to its column nameNoice!
okay, so far so good
how about adding the id?
Wait
since, y'know, we need that
Just trying to see which errors I get when changing the group by values to something that doesnt exist
But ok wll do that now
looks good. not sure why you removed the duration, but.. you do you 😄
also, id is by convention always the first thing you select
Now my next question arrises
How can I get them into the list instead of a string with the values comma separated?
like...
['Family', 'Horror']
?Yes
you'll need to split it in C# then
SQL can't return things results with lists inside a single result
honestly, at that point you might as well just not group the results and just iterate over the results as you go
No that's good enough for me information-wise.
Must be faster iterating over the string and pushing it into the list
¯\_(ツ)_/¯
no need to iterate and push
just... assign
Genres = genresFromSqlAsString.Split(", ")
also, probably want to change that separator
in case a genre ever contains ", " in the name 😄
maybe use a |
or somethingYeah I'm lost again
Thanks for the help though, appreciate it so far 🙂
I'm going to have dinner and get back to it later.