Will it ever be possible to Organize the Internet

[Robert Zwilling]

Things haven't changed at all except for the scale, every year there is more stuff, fiction and facts mixed together, just dumped in a bunch of big piles. The library system works but most physical libraries are outdated except for the tables with all the computer terminals on them. Smaller libraries can't get enough new stuff, they follow trends, and they keep moving the older stuff farther and farther away from the physical location until it just disappears and becomes an online only item. I started using libraries in the 50s. The first school I went to the main entrance was the library. You had to go through the library to get to the rest of the school. The amount of information that is available today is phenomenal but it takes a lot of work to sort everything out. Maybe a vr headset that showed hundreds of pages laid out in a panoramic view where you could flip through the pages would work.

 

[Swank]

Very true, but the same could be said of the entire bulk of published material. The majority is garbage. You just have to be aware of your sources and double/triple check everything. But again, the same could be said of published paper information. And this has always been the case; just take a look at all the Victorian published spiritualism material that was frequently treated as scientific despite a complete lack of solid evidence.

I really think that the only thing that has changed significantly is the scale!

I think there is a difference between bad research and no research at all.

 

[Vertigo]

I think there is a difference between bad research and no research at all.

As has always and will always be the case. I still see no difference today other than the scale.

 

[Dave]

I have found a more specific problem, which may well be replicated elsewhere, and which is what I meant earlier. I'm interested in Y-chromosome genetic genealogy, and it is constantly developing field in which new gene sequences are found and reported, older sequences are updated, and the entire organisational structure of these sequences, based upon the time in the past at which the mutation occurred is constantly updated by committees. If you search for a gene location online, you will undoubtably get old and out of date information because the top of the pile will be the most popularly viewed piece, which will be back when this was most discussed, but not how it is generally regarded at present today. Instead of a current scientific viewpoint, you will get five or ten year old suggestions of what might be true. The classification structure is going to be that which was around the longest, and not the latest produced. Some of the numbers and letters get reused again for totally different sequences which can make things even more confusing still. The bottom line is that using a search engine to look these things up is practically useless. What's more, any AI answer that relies upon searches to produce it's answers is also go to be garbage too.

Now, I have no idea if other areas of research would be the same, but it sounds likely to me. It isn't a case of bad research. The research was good at that time it was produced, but the world moved on, and the internet keeps everything - all the old copies of it's work which it still references when they ought to have been shredded.

 

[Vertigo]

Yes I can see that being a problem. In the physical world the older a piece of research is the more it moves to the bottom of the pile. Whereas, as you say, in the internet old research is effectively treated with the same, if not more, relevance than the new. At least until the new builds up some history. I guess what you need is a search engine that gives greater weight to the most recent stuff. Obviously, this will not automatically give perfect results, there will always be some new, bad research, but maybe will favour the more recent stuff.

Interestingly it seems Google does this for news searches (which makes sense) but it's a shame the user cannot get such weighting on demand:

The weight applied to each factor varies depending on the nature of your query. For example, the freshness of the content plays a bigger role in answering queries about current news topics than it does about dictionary definitions.

 

[Swank]

As has always and will always be the case. I still see no difference today other than the scale.

The difference being - bad research tells on itself, while just insisting that monkeys can fly requires nothing more than repetition and exclamation marks.

 

[Swank]

I have found a more specific problem, which may well be replicated elsewhere, and which is what I meant earlier. I'm interested in Y-chromosome genetic genealogy, and it is constantly developing field in which new gene sequences are found and reported, older sequences are updated, and the entire organisational structure of these sequences, based upon the time in the past at which the mutation occurred is constantly updated by committees. If you search for a gene location online, you will undoubtably get old and out of date information because the top of the pile will be the most popularly viewed piece, which will be back when this was most discussed, but not how it is generally regarded at present today. Instead of a current scientific viewpoint, you will get five or ten year old suggestions of what might be true. The classification structure is going to be that which was around the longest, and not the latest produced. Some of the numbers and letters get reused again for totally different sequences which can make things even more confusing still. The bottom line is that using a search engine to look these things up is practically useless. What's more, any AI answer that relies upon searches to produce it's answers is also go to be garbage too.

Now, I have no idea if other areas of research would be the same, but it sounds likely to me. It isn't a case of bad research. The research was good at that time it was produced, but the world moved on, and the internet keeps everything - all the old copies of it's work which it still references when they ought to have been shredded.

Aren't there journals for this kind of research? How would a university research department get the latest?


I was curious, so I did a search, then clicked on "Google Scholar" and got this page:

Google Scholar

scholar.google.com

Searchable by year and ordered by either relevance or date. You can turn on citations and click to see them for each entry. Is that failing to produce the kind of results you want, Dave?

 

[Dave]

It isn't really scholarly research. It's amateurs taking tests independently and announcing their results via forums much like this, and previously on Lists and Apple Groups that are now dormant. So, there is no order to it. Ordering search results by date would work, but sometimes work may not have been done in a certain area for some years and no way to know that. Anyway, it's a niche thing that probably doesn't apply elsewhere.

I'm not exactly sure what the original complaint was if not about search engines being poor. You can get better search engines for searching for specific subjects.

 

[hitmouse]

Aren't there journals for this kind of research? How would a university research department get the latest?


I was curious, so I did a search, then clicked on "Google Scholar" and got this page:

Google Scholar

scholar.google.com

Searchable by year and ordered by either relevance or date. You can turn on citations and click to see them for each entry. Is that failing to produce the kind of results you want, Dave?

Google scholar is respectable machine for searching for scientific papers. It does not produce listings in the same manner as standard Google. Some prefer Pubmed, but they both work pretty well in terms of cross-referencing, citations, papers by author, year , etc.
it takes a but of a knack but I reckon that with 3 or 4 sequential searches I can usually find key papers on most subjects. I use it at least once week.