❔ Best web scraping/downloading library?
I saw someone recommended anglesharp, but does it allow downloading full page content (javascript,html,css etc.) or does it download only html and css? I am trying to create a tool that downloads a page that loads with javascript.
24 Replies
I am thinking of just trying python out just for this project because of how much options there are
You can only download the client part of a website.
Most of the times the JavaScript parts are obfuscated.
This is also completely unrelated to C#.
$js
What are you even on about? You can download it with ctrl+s, and a bunch of different addons/methods (singlefile addon for chrome for example).
But i haven't found a single way to do it in C# for the program that i was making
I am trying to create a tool that downloads a page that loads with javascript.
?
The tool is in C#
The page loads the html with javascript
Then you have to rephrase it.
Just because you didn't understand it doesn't mean that it's my fault
Because you're saying that you want to download a page with javascript on load.
Yes the page is loaded with javascript
What?
It doesn't have to be.
The html contents are loaded with javascript
Are you referring to a specific website?
I am refering to a page that loads the html contents with javascript
You can't do it normally with httpclient since the javascript needs to execute itself
Is that a requirement?
Yes
What if the website is static without any JavaScript?
Then i wouldn't be here asking for help
So you're saying that you want to download all the resources from a website?
Yes
Alright.
Web Scraping is tedious.
You're most likely going to need something like a WebDriver.
Was this issue resolved? If so, run
/close - otherwise I will mark this as stale and this post will be archived until there is new activity.