The danger of believing too much in AI’s answers | The correct AI answer、Reasons for making bad decisions

With AI、Always get a rational and correct answer。

Someone who feels that way、I think it's increasing little by little.。

actual、Current AI is very useful。write sentences、organize information、give ideas、You can compare multiple options。Even if it's a specialized topic、If you enter certain conditions、Gives a coherent answer in a short time。

I myself use AI on a daily basis.、I have no intention of denying that ability.。

But recently、While involved in a project、When using AI for decision making、I realized a very important issue。

it is、

Even though the AI's answer itself is not wrong.、depending on the answer、What we have been building over time、can be easily rejected

That's what it means。

This is not a story about "AI cannot be trusted".。

rather、I think this is a problem that arises because AI is so good.。

AIは、I don't know the amount of time that person has accumulated.

when proceeding with a project、Even if it looks like one task from the outside,、Behind the scenes, many judgments are piled up.。

Decide on a name。

Unify the notation on the web。

Make information easy to search。

Think about the connection with SNS。

Organize the content to be sent。

Think about who you will send information to in the future.。

Create the necessary materials。

In anticipation of future developments、determine current selection。

These things、If you just look at each one、It may not seem like such a difficult task。

But in reality、While relating these to each other、build up in one direction over a long period of time。

In the process、There is also a reason why we did this.。

I didn't just decide based on my preference.。

There is also a reason why I abandoned the options I had previously considered.。

Once you change what you have decided、Other areas may also be affected。

In other words、Behind what appears to be complete、There is an accumulation of invisible judgments.。

However、That's where AI comes in。

AIは、I haven't watched the project since it started.。

Who decided what and why?。

What problems have you overcome?。

at what cost、What have you prioritized?。

What options did you consider?、Is it in its current form?。

I don't know the history behind it。

What AI knows、fundamentallyInformation entered by the consulter at the timeis。

If you don't understand this、something very dangerous happens。

“Answers that match the wishes of the person”、not necessarily fit for purpose

AIは、Respond while respecting the wishes of the person seeking advice.。

this is natural。

for example、

"I don't want to do this"

If you consult with、The AI ​​said, “Okay.、"What can I do to achieve my goal without having to do anything?"。

“I think this method is suitable for me.”

Speaking of which、It will explain the advantages of the method and how to implement it.。

This is very convenient for consultants。

but、There is one pitfall here。

the person's wishes and、The purpose of the project is、not necessarily match。

For example、Suppose that the purpose of a certain activity is to "get as many people as possible to know about it."。

on the other hand、The person himself has a desire to not appear in public if possible.。

The work has been put in the adult category because of the photo showing the bust of the cut.、

“How to spread awareness without appearing in public”

If you consult AI、Of course AI、think of a method in that direction。

Correct as an answer。

but、

“The method is、Is this the best way to spread awareness?”

That question is different.。

here is、There is a trade-off between the person's wishes and goals.。

AI aligns with the desires of the person seeking advice、Maximizing the overall purpose of the project、not the same。

Even if AI can become “an ally of the counselor”、Not “the person in charge of the entire project”

I think this is very important。

AI can organize the thoughts of the consultant。

You can also raise opposing opinions.。

You can also point out risks。

but、The AI ​​is not responsible for the entire project.。

That's why、When the consultant asks a question about a certain part、Even if the AI ​​returns a reasonable answer for that part、It may not be correct for the entire project.。

For example、Certain changes will bring benefits to the individual.。

but、With that change、Consistency with other parts that have already been constructed may be broken.。

If AI doesn't know that much、its impact cannot be assessed。

This is not a lack of AI ability.、Problems with the scope of information providedis。

“This is what the AI ​​says”、Not a basis for decision making

recently、There is a way to use it that makes me a little concerned.。

Talk to AI about your thoughts、look at that answer、

“AI also says this”

is to think。

of course、There is no problem in referring to AI's answers.。

The problem is、Before you know it、

“Because the AI ​​said so.、My thoughts are correct.”

This will change the perception that。

AI is not a judge。

It does not determine the final correct answer.。

For the entered conditions、It's just generating a plausible answer.。

Moreover,、If the conditions include the consultant's own beliefs and wishes,、Answer based on that premise.。

the result、I just had my thoughts confirmed by AI.、I feel like I have been objectively recognized by a third party.。

I think this is very dangerous。

Instead of “asking AI what you think”、"Use AI to question your own beliefs"

If you want to use AI effectively、You should change the way you ask the question.。

“Are you okay with this idea?”

Not only、

“What are the weaknesses of this idea?”

“What does it look like from the opposite perspective?”

``If this judgment fails,、What could be the cause? ”

“If we maintain the current policy and if we change it、What risks does each pose? ”

I ask。

More importantly、

"This judgment、Is there an assumption I'm overlooking? ”

to ask。

If you do that、AI is not an entity that confirms your ideas.、Become a tool for validating your own judgment。

I、I think the value of AI can be brought out more by using it that way.。

Rather than accumulating、it's easier to break

it takes time to make something。

Create a website。

organize information。

Create a state to be searched。

Operate SNS。

continue sending out messages。

Unify name and design。

Decide on rules with future development in mind。

These things、It is not something that can be completed in one day.。

but、it's easy to break it。

change one decision。

change what has been unified until then。

change direction in another direction。

That's all、The consistency of what you have built up may become inconsistent.。

Moreover,、Immediately after breaking、The effects are often invisible。

That's why I'm scared。

Just a momentary feeling that “things look better now”、A few months after that change、We cannot predict what will happen in a few years.。

Be wary of “small success experiences” caused by AI

One more thing、There are things to be careful about when using AI.。

Asking AI、You'll get an answer right away。

and、Trying out the answer、Sometimes things go well。

The sentences were written clearly。

I got a good idea。

Problem solved immediately。

This will be a successful experience。

Then、

“If you ask AI, you can solve it.”

The feeling of。

While this is convenient,、It's also dangerous。

because、What AI is good at、organize the given problem、to submit a candidateThat's why。

What to set as a problem。

What are the prerequisites?。

in the first place、Do we need to solve the problem now?。

How will this affect existing plans?。

these are、humans must decide。

Even if the AI ​​gives the correct answer、What if the question you should have asked is wrong in the first place?、As a result, you may end up going in the wrong direction.。

There is a difference between “correct answer” and “correct decision making”

this time、It was this difference that I felt most strongly.。

AI's answer is correct。

at least、It was reasonable within the given conditions.。

but、You can accept that answer as is.、may not be right for the entire project。

this is、

Rather than “the AI ​​made a mistake”、“I made a mistake in the scope to which the AI ​​answer was applied.”

I think the problem is。

AI is a very powerful tool。

but、Tools have a purpose。

camera can take great pictures。

but、You can't let the camera decide what to photograph.。

In the same way、AI organizes huge amounts of information、provides food for thought。

but、

“What is your purpose?”

“What are we protecting?”

“What should I throw away?”

“To what extent will we maintain the gains we have made so far?”

If we leave the final decision to AI,、The story will change。

The more you use AI、Human judgment becomes important

The more AI evolves, the more、people don't have to judge。

Some people may think so。

I think it's rather the opposite。

As AI becomes better、

“Which answer should I adopt?”

Human judgment becomes important.。

AIは、can give multiple answers。

It is up to humans to choose one among them.。

and、That choice comes with responsibility.。

especially、For things that have already been built over time、Just because there are new answers、There is no need to immediately negate the previous policy.。

To begin with、

“This new proposal、Doesn't it contradict the existing policy?

“What do you lose if you change?”

“Will changing really get me what I want?”

need to think about。

AI will give you the “answer”。However, you must provide the "context" yourself.

in the end、AI's greatest strength and weakness lie in the same place.。

AI can handle a huge amount of information。

but、If you don't give that information、i don't know。

Judgments made by humans over a long period of time。

the intention behind it。

past experience of failure。

Previously considered options。

relationships。

future goals。

Those things、If it's not in the question, it doesn't exist in the AI.。

So when consulting AI、

“This AI、How much do you know about this issue?

need to think about。

and、If there's a big part you don't know、The answer should not be used as a final decision.。

What is needed in the AI ​​era is not “the ability to believe in AI”

It's not that I don't trust AI。

rather、I trust it a lot。

That's why、I don't think we should leave everything to AI.。

AIは、It's a great tool for thinking。

but、Whether to adopt the answer given by AI、humans have to decide。

especially、If you're changing something that someone has spent a long time building.、

“Because the AI ​​said so.”

not、

“Why change?”

“What do you have to lose?”

“Will the change really increase the likelihood of achieving the goal?”

it is necessary to think about。

It’s not enough to know whether the AI’s answer is correct or not.。

the answer、now、Is it correct to use it in this situation?。

It's the first time I've thought that far、I think AI will be used to make decisions.。

Through this experience、I felt that way again。

AI is right。

nevertheless、If you use it incorrectly、results go in the wrong direction。

and、AI is not responsible for this.。

The final decision is、After all, we are only human.。

Information referenced by AI、not necessarily correct

One more thing、There are things you must remember when using AI.。

The amount of information handled by AI、Whether that information is correct is another matter.。

I have been taking photographs for 50 years.。

If you have been doing photography for a long time、About photography techniques and equipment、There are things you can only know if you have actually tried it yourself.。Even if it's true in theory、Sometimes things don't turn out exactly like that on the actual shooting site.、vice versa、From many years of practical experience, I can sometimes judge that ``this cannot be used under these conditions.''。

However、on the internet、For those who haven't been into photography for that long、In some cases, they provide conclusive information as if they have years of experience.。

of course、Due to short years of experience、This does not mean that the person's information is necessarily wrong.。

The problem is、Even if the information has not been sufficiently verified,、If it spreads widely on SNS etc.、Becoming information that is referenced by many peopleis。

and、When AI learns and references such information、Even if it is not simply a judgment that ``it is correct because there is a lot of information,''、Information widely distributed on the Internet、It may appear as material for an answer.。

You need to be careful here too.。

For example、About a certain photographic technique、Suppose that information sent by an inexperienced person spreads widely on SNS.。

Even if the information actually only holds true under some conditions,、It spreads as “correct information” on the internet.。

after that、When you ask AI about the same topic、There is a possibility that an answer that is consistent with the information circulating on the internet will be generated.。

Then the user、

“AI says the same thing.。It’s probably right after all.”

I think so。

but、this is a very dangerous cycle。

Information transmitted by humans spreads。
AI refers to disseminated information。
AI will give an answer that includes that.。
People who see the answer think, “AI says so too.”。
and、The answers will further strengthen human information dissemination.。

If this cycle occurs、Regardless of whether the initial information is correct or not、“Information that appears to be correct” may be strengthened.。

That's why、When you see the AI's answer、

“It’s true because the AI ​​says so.”

not、

"That information、What kind of evidence is there?”

it is necessary to think。

especially like the photo、In fields where results vary greatly depending on actual experience and shooting conditions.、What is widely talked about on the internet、What actually happens in the field does not always match.。

50Having been taking photos for 2018, I feel that、This is exactly the part。

AI is very useful。

but、The answer given by AI、When what you know from years of experience is inconsistent with what you know、Must stop at least once。

There is a lot of information、Information being correct is not the same thing.。

and、What the AI ​​answered、What is true in the real world、not necessarily the same。

Because this is the age of using AI、It's not about the "amount" of information.、An eye for “evidence” and “context”、more important than ever。