This makes me so happy. I’ve been wanting to start a project to scrape fandom for a while so I can just have the info without accessing their godawful website. Any time I need info on something in a game I get hit with 5 results from fandom and some half-related articles in the search results.
As someone who has tried that effort to do scraping from Fandom, it’s certainly an experience - the HTML is truly nightmarish and definitely written in a way to make scraping as hard as possible (and potentially even programmatically obfuscated, too)
This makes me so happy. I’ve been wanting to start a project to scrape fandom for a while so I can just have the info without accessing their godawful website. Any time I need info on something in a game I get hit with 5 results from fandom and some half-related articles in the search results.
As someone who has tried that effort to do scraping from Fandom, it’s certainly an experience - the HTML is truly nightmarish and definitely written in a way to make scraping as hard as possible (and potentially even programmatically obfuscated, too)