Carl Murray
Web scraper with a database and WebAPI + MVC - help needed
I'm working on a project which requires me to use ASP.NET Core to make a WebAPI, and also a completely seperate MVC app which consumes the WebAPI. It's a college project and these are the requirements which must be met.
To achieve this, I want to build a web scraper which scrapes job ad info for software development related jobs, and displays the demand for various technologies to users to guide them on what they should focus on learning to suit the job market.
So as far as I understand, there are three components to my project:
- Web scraper (written in C#)
- WebAPI service
- MVC application serving the front-end which uses data from the above API
What I need help with:
I do not know how to structure my project so that all of these components work together nicely. For example, I was thinking I could make a .NET console app, add Entity Framework Core and then setup a "Job" and "Technologies" models which have a many-to-many relationship. The web scraper scrapes the data and adds it to the database via EF.
But then how do I connect the WebAPI service to that same database? And how do I setup models in my WebAPI project without duplicating the models I created for the scraper? Can these projects share the same models?
Or should my WebAPI project handle all of the models and database connection, and then the scraper just sends POST requests with the scraped data to the WebAPI?
Would really appreciate any advice on how to best go about this!
5 replies
❔ € symbol prints to console in Rider, but not Visual Studio
Hey all,
Sorry if this is a stupid question, but I'm using Rider IDE on Ubuntu and working on a college assignment.
The lecturer uses VS and will be grading our work based on correct output showing.
My program involves printing the euro symbol € to console. It works fine by default in Rider by just using
Console.WriteLine("€")
However, I tested it in VS on Windows, and the € symbol loads as '?' instead.
I have fixed this by adding Console.OutputEncoding = System.Text.Encoding.Unicode; , and it now works in VS on Windows
But my questions are:
1. How do I know that this method will work all the time, for all operating systems and computers etc.?
2. Is there any standard practice/guideline I can implement in my C# programs to ensure the output prints as expected regardless of the system being used?
3. Why does it work fine in Rider without the above line of code, but not in VS? What is Rider doing behind the scenes that VS isn't doing that makes it require the above line?
4. Should I be including the above line in all of my programs?11 replies