Search engines have become increasingly useless overtime due to the gaming of search engine algorithms, otherwise known as SEO (or SEO-spam). To those unfamiliar, SEO or Search Engine Optimization is the process of gaming search engine algorithms so that your (or your clients) content increases in search ranking and indexing organically as opposed to paying for in-result ads. This isn't new by any means, but lately (the past couple of years or so) I've noticed the overall quality in search results have been rapidly declining with completely useless content.
An example of SEO is what's known as keyword stuffing where popular keywords (ie search terms) are loaded into the webpage, either by inserting words into meta tags, visible content, or backlink anchor text, to increase the likelihood of appearing in search results. Keyword stuffing has been cracked down on, but over the last several years it's become more sophisticated in that the keywords are "stuffed" into what looks like ordinary content, when it's really unnatural, low quality, poorly written content designed solely to get traffic to the page. Why would anyone want this? Primarily for ad-revenue, marketing, or even swaying public opinion on a certain topic.
As a result of these sorts of shady practices, search results have been getting worse and worse. Just take for example trying to find a simple recipe or trying to get an answer to a very simple question. It's become progressively difficult to find recipes or travel recommendations because of all the SEO optimized trash websites. This observation probably isn't news to anyone. If you're reading this think back to the last several times you've used a search engine and recall how the top several websites are typically garbage results. I think that anyone who has used a search engine in the last few years has noticed the quality of search results has become complete and utter shit.
Much of what you get with a modern search engine is heavily SEO optimized with poorly written articles on content farm blogs that are gamed specifically to generate ad revenue or market shit products. There is simply too much commercial smog thrown up by Google, and although DuckDuckGo is my preferred search engine with slightly less crap, it too often yields very low-quality results when I search for specific topics of interest. I hope to get more text-based content, but instead I get bloated websites that's been poorly put together with giant image files and JavaScript. There are no useful insights being offered and will often contain Amazon affiliate links to products that are no longer even available.
I've had similar frustrations with YouTube search. Thanks to uBlock Origin, I've since tweaked YouTube itself to be more tolerable, however the search engine leaves a lot to be desired. Trying to research even a basic topic often gets you product reviews and unboxing videos. YouTube has become the modern day home shopping network. If this wasn't frustrating enough, YouTube will also inject unrelated "popular" videos about 5 - 10 results in. This behavior if absolutely a dark pattern as it derails you from what you were originally searching for so that you will stay on the site longer, but I digress.
A couple search engines I've been impressed by is search.marginalia.nu and wiby.me. Both of these search engines are by no means going to be helpful for everyday use, but search results emphasize lightweight, text-based, non-commercial content. The about section of search.marginalia.nu states “Remember when you used to explore the Internet, when you used to discover cool little websites made by people and it wasn't just a bunch of low effort content mill listicles and blog spam? I want to show you that the Internet you used to go exploring is still very much there. There are still tons of small personal websites, and a wealth of long form text from both the past and the present. ” Wiby expresses a similar sentiment on their about page: “In the early days of the web, pages were made primarily by hobbyists, academics, and computer savvy people about subjects they were personally interested in. Later on, the web became saturated with commercial pages that overcrowded everything else. All the personalized websites are hidden among a pile of commercial pages… The Wiby search engine is building a web of pages as it was in the earlier days of the internet. In addition, Wiby helps vintage computers to continue browsing the web, as pages indexed are more suitable for their performance.”
So what's the solution? I think at least some of this SEO-spam can be mitigated if search engines allowed users to permanently block entire domains that never contain any useful information Pinterest, Quora, etc, and any other website that I (the user) deem as spammy and useless content (this can technically be accomplished now with the "-" operators to exlude words and domains, but 1) this requires the user to enter this for every single search query, and 2) search engines are increasingly not respecting the exclude operator). I also think prioritizes not just text-based content but also lightweight websites in not just overall size, but sites with little to no tracking, excess JavaScript, bloat, etc could also curb it a bit and therefore make search results more useful. I have been using a browser extension called uBlacklist which adds the ability to block websites from search results by injecting "Block this site" into the results. It's come in handy, however there's so much garbage websites gaming the system, it's an impossible task to block them all from any conceivable thing I may search for using a seach engine.
Thanks for reading. Feel free to send comments, questions, or recommendations to hey@chuck.is.