How would you test intelligence?

Published on Dec 29, 2019

How is intelligence tested?

If you were given the task to design an intelligence test how would you test someone’s intelligence?

Let’s look at a hypothetical scenario, there are 2 people to be tested, each person puts together an intelligence test for one another. Each person is allowed to test the other on any subject they like but they must know the answer themselves. One person has been brought up in the Amazon rainforest and the other was brought up in a European capital city, will their tests be fair? Will their tests be a true accuracy of intelligence? No, their tests will be relative to their environments and upbringing……this is known as bias.

If you take a standard IQ test, this test doesn’t provide a true mark against one’s general intelligence, all this test provides is an indication of someone’s intelligence relative to their environment and how they performed on that day with that particular test and set of questions. An IQ test in China would be different to an IQ test in England despite the difference in language. If you can do long division in your head but you can’t change a wheel on a car and vice versa, what does that make you? What is intelligence anyway?

TonkaBI have developed an Ai computer vision application to analyse images of vehicles, the Ai looks for damage on the images of vehicles (amongst other things ). Would it be fair or correct to test this Ai intelligence on images of fruit or motorbikes? The answer is no, it would not be fair or an accurate way to test the intelligence of this Ai. Again, intelligence is local to the environment. A hunter in the Amazon wouldn’t be considered intelligent if he could not differentiate between poisonous and none poisonous food.

The race to develop the next best Ai is on! But testing Ai can be a little tricky. Also remember that Bias – we are seeing lots of examples where the Ai is developed only for it to fail owing to built in Bias.

Your company has given you a task, you need to test a chat bot Ai, how do you do it?

Let’s assume this Ai has a high capability level around speech and language, plus the Ai understands your culture, the country you’re from and is fluent in your native language; so it seems pretty smart. The Ai also understands human behaviour (this Ai sounds like it could be a pretty good customer service bot!).

This is where the Turing Test can be brought in. Your task would be to develop a suitable test to measure the bot’s intelligence and ability to interact with humans. How would you test this Ai? Merely asking the Ai a couple of questions such as, who is the president or prime minster? Or, what day of the week is it, would not suffice. You need to understand the Ai’s general intelligence capability, so maybe an emotional or conscious response would be sufficient?

But how would you get such a response from an Ai?

Alan Turing proposed a test called “imitation game” where a series of questions would be asked by a human and the answers would then be assessed. The idea of the game is to test an Ai against a human and study the responses given by both parties. The judge or assessor would have to decide which answer was given by the Ai. If the judge selects the Ai then the Ai wins, this is now been called “The Turing Test”. Basically, an AI needs to fool a human in to believing the answer was given by another human.

What would be your Turing Test for this bot?

Posted in blog on Dec 29, 2019