A.I. (general thread for any AI-related topics)

[Vertigo]

I doubt we will all suddenly rush out and buy self-driving cars, just as we aren't going to buy AI written books. However, just as we use grammar checkers and other AI to revise and edit, new cars are being built with sensors to automatically prevent such rear-ending. My automatic car with cruise control and lane control already leaves me with little to do. I expect that other incremental changes will eventually result in a fully self-driving car without anyone noticing that it had happened.

This is I think exactly how things will progress and, of course, it is a technology very much still in its infancy and as such will inevitably attract very polarised views. I am convinced that self-driving cars will eventually completely displace human drivers. Their safety record will eventually become so much better that it is inevitable.

And let's be honest here it is a very low bar. Take a look at the statistics for car accidents. We are a race of (mostly) amateur operators of pieces of machinery weighing 1 to 2, or more, tons travelling at eye watering speeds. I can't think of any other regulated, commercial mechanical process with such high levels of danger being permitted with so little training. There were some posts here a while ago stating that in some states of the USA you can get a car licence with pure theory!

However, apologies, I don't want to derail a very good thread to the politics of car driving!

 

[CupofJoe]

Top Gun: AI

What will future aerial dogfights look like?

AI with dogfighting ability is being developed by the UK and US air forces.

www.bbc.co.uk

 

[Ursa major]

so it could then 'see' beauty etc..

Seeing it is not the same as being able to appreciate it... let alone having the feeling one can get when doing so.

 

[Robert Zwilling]

It didn't see the stop sign or it ignored it. It didn't see the flashing lights or it ignored them. It didn't sense the car behind it or it ignored it. I doubt it didn't see or sense those things. People with impaired thinking can ignore or not see these things. It could be that for whatever reason the program has reached a point where it can't take in any new information or handle any new information, so it prioritizes what does get processed. The program is gliding through space, possibly not using unusual pieces of data but instead relying on things being normal where responses are done automatically. No way of telling how much is being handled live and how much is just fingers crossed. Ironically, if the program crashes and the vehicle has to stop moving, and does so in traffic or in an inopportune location the software crash can cause a physical crash. Perhaps an example where the fourth wall is broken by theatrical technique.

 

[Robert Zwilling]

The microsoft cloudburst outage shows another way AI could take over, by turning everything off and maybe not turning it back on again. Southwest airlines remained in business because their basically using windows 3.1

AI enhanced answers seem to be combining the very highly probable with the highly improbable but still true into one statement. Such as saying potassium is a naturally occurring radioactive element. There ae three versions of potassium, two are not radioactive, the third is radioactive and composes .012 percent of naturally occurring potassium.

 

[Vertigo]

The microsoft cloudburst outage shows another way AI could take over, by turning everything off and maybe not turning it back on again. Southwest airlines remained in business because their basically using windows 3.1

Sorry, but the recent IT 'outage' had nothing to do with Microsoft and the windows 3.1 stuff is apocryphal rumour spread from one report describing their systems as looking like they were designed on Windows 95. The problem was not with the operating system it was in an update from a company called CROWDSTRIKE (not cloudburst), an antivirus and security software company. Businesses that were unaffected (and there were many) were unaffected because they were not using the Crowdstrike software, not because they were using a particular operating system. Apple and Unix were unaffected because the bug was not in updates to the software running on those system (if they even have Apple and Unix versions).

However I would agree that the IT world faces something of a dilemma. On the one hand having almost all IT running on a very few different operating systems means a high degree of consistency which is hugely convenient, but on the other hand any problem with one of those systems can have a massive impact across the globe.

However work-arounds for such problems are almost invariably very rapidly produced (much more so than other global threats such as pandemics) and, before anyone gets into major AI paranoia, no one person, AI, computer or anything else is capable of 'turning everything off' and certainly not capable of keeping it turned off. We are still a very long way from the world of Skynet; the internet is a decentralised network with no single entity that controls its entire operation. A little perspective is needed. This bug from Crowdstrike was bad, and shouldn't have happened, but it only impacted a bunch of major companies for less than a day. The blocking of the Suez canal (through human blundering) 3 years ago had a much bigger impact on global economy than this could ever have done.

I repeat the whole business about an airline company still using Windows 3.1 can be traced back to a joke tweet and article, which every news outlet has picked up on because knocking Microsoft has always been a good news seller. Who was affected and who wasn't had nothing to do with the operating systems they were running and everything to do with the anti-virus security software they were running. As to why it could happen (of course it never should have if the update had been properly checked but that's another story), you must realise that security software has to have extremely low level access to operating systems in order to combat viruses and hackers. This means that bugs can effectively do as much damage as the viruses they are meant to protect against.

 

[Robert Zwilling]

Cloudburst is like fakebook, intended.

 

[Foxbat]

AI to be intergrated into Spear Cruise Missiles.

MBDA planning AI integration for SPEAR cruise missiles

The company assures that the system will ensure the missiles operate within the boundaries set by human operators, maintaining legal and ethical standards.

ukdefencejournal.org.uk

Reading the article, it sounds like the main use of AI will be to enable better evasion capabilities, allowing the missile a greater chance of reaching its target.

 

[Mon0Zer0]

The "AI" that exists does not do any of that.
Existing "AI" is a vast database of pre-processed disjointed "phrases" that it recombines in response to a prompt. If you ask about "the Sermon on the Mount" current "AI" systems doesn't read respond to your prompt by reading all the versions of the bible and then providing a reasoned analysis. "AI" searches it's database for the phrase "Sermon on the Mount" and then looks at all the other extended phrases in its database that includes the term and creates a coherent amalgam of those stored phrases.

Here is an excellent discussion on what AI is and what it isn't:

Collections: On ChatGPT

So I stirred up a bit of conversation on Twitter last week when I noted that I had already been handed ChatGPT produced assignments. For those who are unaware, ChatGPT is an ‘AI’ chatbo…

acoup.blog

Here is an outtake:
ChatGPT is chatbot (a program designed to mimic human conversation) that uses a large language model (a giant model of probabilities of what words will appear and in what order). That large language model was produced through a giant text base (some 570GB, reportedly) though I can’t find that OpenAI has been transparent about what was and was not in that training base (though no part of that training data is post-2021, apparently). The program was then trained by human trainers who both gave the model a prompt and an appropriate output to that prompt (supervised fine tuning) or else had the model generate several responses to a prompt and then humans sorted those responses best to worst (the reward model). At each stage the model is refined (CGP Grey has a very accessible description of how this works) to produce results more in keeping with what the human trainers expect or desire. This last step is really important whenever anyone suggests that it would be trivial to train ChatGPT on a large new dataset; a lot of human intervention was in fact required to get these results.

It is crucial to note, however, what the data is that is being collected and refined in the training system here: it is purely information about how words appear in relation to each other. That is, how often words occur together, how closely, in what relative positions and so on. It is not, as we do, storing definitions or associations between those words and their real world referents, nor is it storing a perfect copy of the training material for future reference. ChatGPT does not sit atop a great library it can peer through at will; it has read every book in the library once and distilled the statistical relationships between the words in that library and then burned the library.

This is correct, but in creating an n dimensional database of relations between words, it also creates a network of relations between concepts as a by-product. For example LLM's can recognise the relation between Uncle and Aunt and Male and Female because in N dimensional "latent space" occupy positions nearer to their sex based words, enough so that the LLM can discern the relations between them. Concepts arise from these relations as a result. They may not be "concepts" as we intuitively understand them, but concepts are nonetheless emergent properties of their positions in latent space.

It's hotly contested whether this is a form of (proto-)understanding or if the concepts are meaningful to the machine, but it *may* be at least one facet of how we process information and concepts in our own neurons.

Humans don't store "definitions" as you state above, but they do have referents to the real world as part of the web of connections in the brain. The challenge AI researchers are embarking on is building up relations between language and the things themselves in the real world. The intersection of video and audio recognition and LLM's is one aspect of that.

As LLM's become embodied within the world, these relations will be part of the training.

ChatGPT does not understand the logical correlations of these words or the actual things that the words (as symbols) signify (their ‘referents’). It does not know that water makes you wet, only that ‘water’ and ‘wet’ tend to appear together and humans sometimes say ‘water makes you wet’ (in that order) for reasons it does not and cannot understand.

LLM's are getting better at reasoning - you can see that in the improvements in Claude Sonnet, where reasoning was explicit in the latest round of training.

In that sense, ChatGPT’s greatest limitation is that it doesn’t know anything about anything; it isn’t storing definitions of words or a sense of their meanings or connections to real world objects or facts to reference about them. ChatGPT is, in fact, incapable of knowing anything at all. The assumption so many people make is that when they ask ChatGPT a question, it ‘researches’ the answer the way we would, perhaps by checking Wikipedia for the relevant information. But ChatGPT doesn’t have ‘information’ in this sense; it has no discrete facts. To put it one way, ChatGPT does not and cannot know that “World War I started in 1914.” What it does know is that “World War I” “1914” and “start” (and its synonyms) tend to appear together in its training material, so when you ask, “when did WWI start?” it can give that answer. But it can also give absolutely nonsensical or blatantly wrong answers with exactly the same kind of confidence because the language model has no space for knowledge as we understand it; it merely has a model of the statistical relationships between how words appear in its training material.

We lack a proper definition of what it means to "know" something. Plenty of humans, in the wild, believe wacky things or are unable to verify truth. One aspect of the modern age is the rise of "fake news" and "alternative facts" and we all know people who will spout blatantly wrong things with confidence. Ontology is a recurrent philosophical problem.

In artificial intelligence studies, this habit of manufacturing false information gets called an “artificial hallucination,” but I’ll be frank I think this sort of terminology begs the question.4

We have to bear in mind that LLM's and "AI" are in very early stages. Hallucinations are being investigated and researched, as is embodiment. This is the hard challenge that self driving cars have had to take on - as Musk has said, to tackle it properly you need AGI.

 

[Mon0Zer0]

AI to be intergrated into Spear Cruise Missiles.

MBDA planning AI integration for SPEAR cruise missiles

The company assures that the system will ensure the missiles operate within the boundaries set by human operators, maintaining legal and ethical standards.

ukdefencejournal.org.uk

Reading the article, it sounds like the main use of AI will be to enable better evasion capabilities, allowing the missile a greater chance of reaching its target.

Horrifyingly, AI was used to identify targets for air-strikes in Israel-Palestine, recently.

 

[Robert Zwilling]

Humans don't store "definitions" as you state above, but they do have referents to the real world as part of the web of connections in the brain.

Basically that seems to say that humans and AI programs are doing the same thing. Does this mean that humans, unlike AI systems, are correct more often because they don't have billions of free floating facts to choose from to fill in the blanks with. Because people have fewer choices to choose from they have better odds of getting it right?

During a conversation when I forget a word ("term") I can use the definition of the word in place of the word and continue talking. The definition I use is made up on the fly and not stored anywhere?

Plenty of humans, in the wild, believe wacky things or are unable to verify truth.

What does in the wild mean? Everyone has wacky ideas about something. Everyone uses only parts of descriptions of events so they can convey thoughts without having to take all day explaining something. This highlights the ideas they personally endorse or want to emphasize or perhaps only know, which is a partial explanation of what happened, not what actually happened, which would be the true situation.

 

[Dave]

This highlights the ideas they personally endorse or want to emphasize or perhaps only know, which is a partial explanation of what happened, not what actually happened, which would be the true situation.

Interesting point. Truth cannot be defined by a mathematical formula. Truth is different for each person. Ask different football supporters after a game if a goal should have been allowed or not? Same goes for any sport, politics, religion, history (even recent history gets rewritten), how to look after babies, how to eat scones.... because these are not "truths" at all, they are debateable points of view.

I wonder how a true AI will deal with this? When James T Kirk debates it into the endless loop of a logic problem will it still blow a fuse or can it make a decision for itself?

 

[Wayne Mack]

Interesting point. Truth cannot be defined by a mathematical formula. Truth is different for each person. Ask different football supporters after a game if a goal should have been allowed or not? Same goes for any sport, politics, religion, history (even recent history gets rewritten), how to look after babies, how to eat scones.... because these are not "truths" at all, they are debateable points of view.

I wonder how a true AI will deal with this? When James T Kirk debates it into the endless loop of a logic problem will it still blow a fuse or can it make a decision for itself?

Computer systems have been defining truth, actually true and false, for some time. The basic model is that inputs are represented as individual values. Each of those values is then weighted and the result aggregated. Then cut off criteria are used to segment the final value into areas of true and false with an indeterminate range between the two.

A computer system will not go into an endless loop; it will provide an answer. The validity of the answer may be problematic, though.

 

[Cthulhu.Science]

Computer systems have been defining truth, actually true and false, for some time. The basic model is that inputs are represented as individual values. Each of those values is then weighted and the result aggregated. Then cut off criteria are used to segment the final value into areas of true and false with an indeterminate range between the two.

A computer system will not go into an endless loop; it will provide an answer. The validity of the answer may be problematic, though.

Sometimes it will take 7,500,000 years to come to a conclusion. And sometimes that conclusion is 42.

Best to ask the right question.

 

[Robert Zwilling]

In the crowdstrike outage, or cloudburst, 8.5 million window devices were impacted. Not counting losses microsoft might experience, the loss so far is estimated to 5.4 billion dollars. Malaysia's digital minister called on CrowdStrike and Microsoft to consider compensating affected companies. Its doubtful anything will come of that.

The principal problem seems to be that crowdstrike used software to test software. Self policing scenario? Apparently a human did not run the software update in a machine as a final step. The question is did a human actually run any part of the software update in a machine at any point during its conception. How reliant are they on machines for writing and testing their code?

Part of crowdstrikes solution seems to be having more human interaction with the code work but it is hard to see that in the wording of their solutions to avoid a repeat performance. Human might be described as third party, local developer testing, or "A new check is in process to guard against this type of problematic content from being deployed in the future." The last one could be a person actually running the program in a machine to see if the machine is still operational after the update.

Complete reliance on machines to do our work is probably the worst effect AI will have on humanity. People have become part of the data process by providing feedback through their public and private actions. The crowdstrike error probably turned out to be less impactful than it could have been, as the error popped up immediately. Imagine what will happen as bad information is fed out to people over a long period of time where it doesn't trip any immediate shutdowns.

An increasing number of scientific reports are being generated by AI assistance. This is being tracked by the vocabulary being used in the reports. The estimate ranges from 1 percent to 17.5 percent. Computer science papers are the worst, math and nature are less likely. As long as the computer industry is bent on reducing the human participation in the workforce, problems will only continue to escalate.

As the electronics industry has plowed forward by making more advanced equipment available at a lower price, the medical industry has gone in the opposite direction by incorporating more expensive processes to replace older processes. The computer industry is shedding jobs at the company level while boosting employment at the individual level and in non technical jobs assisted by computers. The medical industry is boosting employment by needing more and more people to interact between the machines and the people, which includes patients and other workers in the medical field.

In a lot of situations, people are taking a back seat to machines as the machines do a faster, more complex job for less money. It is assumed that the machines don't make mistakes but there is an error factor that is not accurately accounted for. Common searches only yield results about computational errors and bad data entry, which are very low. This doesn't take into account the errors generated by faulty assumptions made by the program designers or the use of data which passes the accuracy test as being true, but is the result of flawed process that generate "bad" but verifiable results. Its looking more like a science fiction story with the queen being a machine and the humans are drones, soldiers, and workers.

 

[Mon0Zer0]

Basically that seems to say that humans and AI programs are doing the same thing.

Sort of. Information is stored in the brain in an analogous way to weights in an LLM by connections of neurons. It's not identical as human neurons are far more complicated and use chemicals etc and have more functions. But information is effectively stored in a multidimensional array, in such a way that relations form that give rise to concepts, behaviours. Different parts of the brain are also specialised to perform specific tasks like sight, breathing, releasing hormones and so on.

Does this mean that humans, unlike AI systems, are correct more often because they don't have billions of free floating facts to choose from to fill in the blanks with. Because people have fewer choices to choose from they have better odds of getting it right?

That might be one aspect, but also that our brains have more filters built in that ensures higher data quality, better storage systems and repeated exposure to phenomena in the real world.

If you've ever talked to someone whose taken LSD, for example, they might describe seeing "tracers" - smeared after images formed from movement. This is a result of LSD disrupting chemicals that filter signals between synapses meaning that the brain is overloaded with visual information.

During a conversation when I forget a word ("term") I can use the definition of the word in place of the word and continue talking. The definition I use is made up on the fly and not stored anywhere?

When I say a definition is not stored, I mean there is no series of "code" (a sentence) stored in the brain that makes up our sole understanding of the thing we're talking about.

The concept of a thing arises from a pattern of relations between different phenomena that make up the concept of the thing. Its shape, its function, what its made of, how we relate to it and so on. Meaning is constructed from facets of the thing. When confronted with the thing we do something similar to an AI in that we identify it based on its attributes, not on its linguistic definition. When we talk about the thing we assign a signifier - a word - to refer to the thing, but the meaning of that word comes after the concept.

In addition, it's impossible to construct iron clad semantic definitions of things that fully encapsulate the thing itself that everyone agrees on. Trying to create a definition of a chair, for example, that doesn't also apply to something else or that captures each example of a chair is impossible. Yet we all know what a chair is.

Our knowledge of the thing comes from experience of the thing itself not our language about the thing. The experience is stored in our brain in a neural connectome -- a web of connections between neurons.

In conversation, we might use a repeated set of words to refer to the item in place of a singular word that could be learned by rote, but even that sequence would still only make up a part of the overall "weight" of the thing itself within the connectome.

What does in the wild mean?

By "in the wild", I meant in real life, haha.

Everyone has wacky ideas about something. Everyone uses only parts of descriptions of events so they can convey thoughts without having to take all day explaining something. This highlights the ideas they personally endorse or want to emphasize or perhaps only know, which is a partial explanation of what happened, not what actually happened, which would be the true situation.

Definitely. Our experiences with the things themselves does help to reinforce ideas that more correspond to reality than an AI which is cast adrift in virtual space with only language on which to form ideas. Hallucinations in humans, however, are moderated by physical and social reality. When we have little experience to drawn on, our hallucinations can get quite far out - as we see with the entire history of Sci-fi and fantasy!

 

[Mon0Zer0]

Complete reliance on machines to do our work is probably the worst effect AI will have on humanity.

This is very true - and has been known for some time in the aviation industry where the use of autopilots has lowered the skill level of pilots and made them unable to react to failure.

The Impact of Automation on Airline Pilots - AAG Philippines

Explore how automation is reshaping the role of airline pilots. Discover the challenges and advancements influencing the future of aviation.

aag.aero

 

[Wayne Mack]

The principal problem seems to be that crowdstrike used software to test software.

This is currently considered best practice (a pipeline or sometimes a CI/CD pipeline) as it is far more intensive and repeatable than human testing. It is a little unclear to me as to where the fault occurred. Either the testing system failed to detect a dead machine after an install or the system detected the failure and did not abort the release at that point.

One of the weaknesses in many pipeline systems is that they are not recursive. There is no automated test of the pipeline to determine whether it is operating as intended. If I were to guess, the failure occurred because there were multiple parallel paths for various operating systems and versions and the logic did not fail the release when one path failed and the other paths passed.

 

[Robert Zwilling]

The Crowdstrike outage, although I like the sound of cloudstrike, shows how AI is used to replace people. There are a lot of different window configurations in a lot of different machines. There is a lot of overlapping but I guess if people were to manually test a software update by literally installing it in most every conceivable set up, that would take a long time or a lot of people and a lot of machines. That's were AI is supposed to shine, to handle multiple situations at the same time, cutting down the testing time dramatically. Only it didn't. It did prove it was only as good as its programming.

Apparently cloudstrike is set up to be loaded as windows prepares to load and if something doesn't go right windows or the operating system in general stops dead in its tracks. I have seen links, 5 or 6 instances, that mentioned previous update failures for other operating systems which were not as spectacular probably because it involved less machines and it wasn't windows. But this time, the number of machines couldn't be ignored. Being windows, everybody likes to throw bricks at windows. Apparently bricks is a computery word used to describe a dead computer.

Another odd word that showed up for the uninitiated was the word dogfooding. It was right there in the official report. " This culminates in a staged sensor rollout process that starts with dogfooding internally at CrowdStrike." It means using your own product. Even if it is true, it's such a strange word. Apparently no one ate the dog food that night. Most of us don't eat our pet's food so why would the term dogfooding coined from a 1970s tv commercial instill confidence if you don't know what the buzz word means. Maybe its supposed to be funny.

Its been speculated that windows could change the way it sets itself up so if a third party essential product goofs, it won't take out windows. That doesn't put much trust in the third party vendors. If its really important such as intruder/vandal protection you wouldn't want the machine running anyway. Although being able to quickly get up and running using the previous setup might not be so bad.

AI can be programed to do only so much. Maybe for really important things we do need people looking over AIs shoulder, a lot of people. Trying to please the shareholder by reducing employees engineers a lot of unintended consequences. A lot of companies hire people to increase their customers interaction and use of their software. It probably wouldn't hurt to start hiring a lot of people to study the consequences of people increasingly interacting with machines more and more. We could call them technical advisers, or maybe philosophers. Once upon a time philosophy was an important subject. Maybe it should be made important again.

 

[Wayne Mack]

The Crowdstrike outage, although I like the sound of cloudstrike, shows how AI is used to replace people. There are a lot of different window configurations in a lot of different machines. There is a lot of overlapping but I guess if people were to manually test a software update by literally installing it in most every conceivable set up, that would take a long time or a lot of people and a lot of machines. That's were AI is supposed to shine, to handle multiple situations at the same time, cutting down the testing time dramatically. Only it didn't. It did prove it was only as good as its programming.

Apparently cloudstrike is set up to be loaded as windows prepares to load and if something doesn't go right windows or the operating system in general stops dead in its tracks. I have seen links, 5 or 6 instances, that mentioned previous update failures for other operating systems which were not as spectacular probably because it involved less machines and it wasn't windows. But this time, the number of machines couldn't be ignored. Being windows, everybody likes to throw bricks at windows. Apparently bricks is a computery word used to describe a dead computer.

Another odd word that showed up for the uninitiated was the word dogfooding. It was right there in the official report. " This culminates in a staged sensor rollout process that starts with dogfooding internally at CrowdStrike." It means using your own product. Even if it is true, it's such a strange word. Apparently no one ate the dog food that night. Most of us don't eat our pet's food so why would the term dogfooding coined from a 1970s tv commercial instill confidence if you don't know what the buzz word means. Maybe its supposed to be funny.

Its been speculated that windows could change the way it sets itself up so if a third party essential product goofs, it won't take out windows. That doesn't put much trust in the third party vendors. If its really important such as intruder/vandal protection you wouldn't want the machine running anyway. Although being able to quickly get up and running using the previous setup might not be so bad.

AI can be programed to do only so much. Maybe for really important things we do need people looking over AIs shoulder, a lot of people. Trying to please the shareholder by reducing employees engineers a lot of unintended consequences. A lot of companies hire people to increase their customers interaction and use of their software. It probably wouldn't hurt to start hiring a lot of people to study the consequences of people increasingly interacting with machines more and more. We could call them technical advisers, or maybe philosophers. Once upon a time philosophy was an important subject. Maybe it should be made important again.

I do not believe that AI was involved in the Crowdstrike debacle. Build and release systems use standard programming techniques, but it seems these were very poorly done by Crowdstrike. It is computer systems that do repeatable things more quickly and more consistently than humans can. Annual releases are no longer acceptable for software programs and this the driving force behind the use of automation.

Aside: "Eating your own dog food" is a long used (for several decades) phrase that simply means that software developers should use their own products. It seems to be a very 'insidery' thing to put in an externally published report. I also fail to see how it is applicable. Security software, such as Crowdstrike, targets multiple operating systems and multiple operating system versions. Given the standardization common for software development machines, it is unlikely that there was any more than a single operating system and version in use by the development team.