✅ Web Scraper

using System;
using HtmlAgilityPack;

class Program
{
static void Main(string[] args)
{
Console.Write("Enter pokemon name: ");
string pokemonName = Console.ReadLine();
string url = $"https://www.pokemon.com/uk/pokedex/{pokemonName}";

var web = new HtmlWeb();
var doc = web.Load(url);

var descriptionElement = doc.DocumentNode.SelectSingleNode("//p[contains(@class, 'version-x') and contains(@class, 'active')]");

if (descriptionElement != null)
{
string description = descriptionElement.InnerText.Trim();
Console.WriteLine(description);
Console.ReadLine();
}
else
{
Console.WriteLine("Description not found.");
Console.ReadLine();
}
}
}
using System;
using HtmlAgilityPack;

class Program
{
static void Main(string[] args)
{
Console.Write("Enter pokemon name: ");
string pokemonName = Console.ReadLine();
string url = $"https://www.pokemon.com/uk/pokedex/{pokemonName}";

var web = new HtmlWeb();
var doc = web.Load(url);

var descriptionElement = doc.DocumentNode.SelectSingleNode("//p[contains(@class, 'version-x') and contains(@class, 'active')]");

if (descriptionElement != null)
{
string description = descriptionElement.InnerText.Trim();
Console.WriteLine(description);
Console.ReadLine();
}
else
{
Console.WriteLine("Description not found.");
Console.ReadLine();
}
}
}
Why isn't this working?
34 Replies
The Connected
The ConnectedOP9mo ago
It works in python with the soup module, but idk why it isn't working in c#
Keswiik
Keswiik9mo ago
Web scraping isn't allowed here most of the time due to it being against the ToS of most sites.
The Connected
The ConnectedOP9mo ago
it isn't for bad intent it's for school idk if it's web scraping, is it? getting info from a website
Pobiega
Pobiega9mo ago
this is most certainly web scraping.
The Connected
The ConnectedOP9mo ago
oh
Pobiega
Pobiega9mo ago
however, that said, I dont think in this case its a huge problem
Keswiik
Keswiik9mo ago
Intent doesn't matter. Still scraping.
Pobiega
Pobiega9mo ago
you are scraping a single pageload, not crawling the entire site
Buddy
Buddy9mo ago
If there is no API then it usually isnt allowed. You can however check the terms and conditions of said site and maybe even robots.txt, although robots.txt is mainly for crawlers and not web scrapers.
The Connected
The ConnectedOP9mo ago
ofc it's for an asp.net web application
The Connected
The ConnectedOP9mo ago
No description
The Connected
The ConnectedOP9mo ago
idk how it is working right no w but before it wasn't working
Pobiega
Pobiega9mo ago
is it mandated that you use webscraping? because there is an excellent pokemon api :p
The Connected
The ConnectedOP9mo ago
i'm not sure what you mean by mandated as in does my teacher know?
Pobiega
Pobiega9mo ago
no, as in does the assignment say you SHOULD use scraping and not fetch the data from an API instead
The Connected
The ConnectedOP9mo ago
no, but i'm trying to learn how to do it with webscraping, it seems easier than sending requests to an api we haven't learned how to use apis
leowest
leowest9mo ago
PokéAPI
An open RESTful API for Pokémon data
The Connected
The ConnectedOP9mo ago
oo
leowest
leowest9mo ago
does this not have the info u want?
The Connected
The ConnectedOP9mo ago
i don't know, what info does the api give?
leowest
leowest9mo ago
open it and check
The Connected
The ConnectedOP9mo ago
we have a selected 800 pokemon database it's safe, right?
leowest
leowest9mo ago
define safe
The Connected
The ConnectedOP9mo ago
it isn't malicious or doesn't take data and keeps my device healthy doesn't do anything out of the ordinary
leowest
leowest9mo ago
it does as much as u were doing with the website above I suggest you open the site and read and see if it fits your purpose or not its way better than webscrapping
The Connected
The ConnectedOP9mo ago
lmao i have no idea what all this is im new to APIs but thank you for your help
Jimmacle
Jimmacle9mo ago
web scraping is never easier than using a proper API web pages are designed for people to read, APIs are designed for computers to read
The Connected
The ConnectedOP9mo ago
that's IF you know how to use APIs i don't do JS
Jimmacle
Jimmacle9mo ago
JS is not required or related to APIs
The Connected
The ConnectedOP9mo ago
that's a good way to put it can u use c#?
Jimmacle
Jimmacle9mo ago
APIs are typically HTTP there is no language requirement
The Connected
The ConnectedOP9mo ago
that's a W then, i'll look into it thank you how do i close
Jimmacle
Jimmacle9mo ago
$close
MODiX
MODiX9mo ago
Use the /close command to mark a forum thread as answered
Want results from more Discord servers?
Add your server