I have been busy traveling and spending time with family and friends so I have not posted in a couple weeks. Sorry about that. Today is January 01, 2018, so Happy New Year. It is and for the past several days has been quite cold in this part of the country. Today we woke up to a fresh -13F. My wife and I will go to the Mall of America in an hour or so to get some steps on our Fitbits.
In the past decade or so there has been a huge interest in artificial intelligence. It seems that, different aspects of AI have been creeping into applications in multiple domains (i.e., Biology, Business, Computer Science, Computer Vision, Economy, and many others).
Not sure if it is a universal definition of AI. In the past few years I have been reading books and experimenting with concepts from AI. It seems like there is a large set of definitions. As usual, when in doubt ask.
To make it simple, I used OneLook (https://www.onelook.com/) and searched the definitions for the word “intelligence”. Following are definitions taken from the top three dictionaries (Merriam-Webster, Oxford Dictionaries, and American Heritage Dictionary of the English 
Language):
The ability to learn or understand or to deal with new situations.
The ability to apply knowledge to manipulate one’s environment or to think abstractly as measured by objective criteria (i.e., tests).
The ability to acquire and apply knowledge and skills.
The act of understanding.
The above definitions are quite interesting but seem to be somewhat vague. In my opinion they are vague because human languages are not precise. A computer language is precise. Let’s take the first definition and go over the words in bold.
I know that two is less than three. I have learned that when I was quite young. I also learned the tables of multiplication for the first twelve integral numbers from one to twelve. I recall memorizing the 144 answers in elementary school. As I am writing this post, I asked my oldest son who, who happens to be working in the room next to my home office, if he or his daughters had to memorize such multiplication tables. It seems that such practice is still in effect. So there you have it, these reduced set of humans, learned (memorized) the first twelve multiplication tables.
I also recall in elementary school, being taught how to write / draw with a pencil, a fountain pen and finally with a ballpoint pen the lower and upper case letters in the English alphabet. They used the Palmer Method. The teachers at my wife’s school used a different method. One way or the other, today my wife and I are able to write and understand / recognize the 26 lower and 26 upper case letters in the English alphabet.
In addition, we both are able to recognize the same set of characters when they are typed using different fonts. I believe we can both take the set of 52 characters, mix and match them and write words in English, French, Italian and Spanish. By the way, some of those languages make use of a few more letters in lower and upper case which we have encountered and have learned how to read and write them in words and sentences. I believe such transposition of knowledge falls into the general area of new situations.
Have you ever watched a human baby learn how to communicate with their 
parents? Most of us have. Babies start by crying because they are not able to express their basic needs (i.e., being hungry, being cold, being hot, being tired, etc). Then the baby recognizes the person that gives them what they need. With repetition and brain development, they start uttering some words and after a couple years of age, they are able to express themselves using a human language (i.e., English). Not only that, but they understand (in most cases and depending on what they are being told) what other humans are telling them. It is certainly amassing what happens in a human brain in about two to three years.
I also took a look at Wikipedia’s entry for artificial intelligence (https://en.wikipedia.org/wiki/Artificial_intelligence). As the entry mentions, AI has been around since 1956.
The Wikipedia article goes into representation of knowledge. I have an issue with such approach. As far as we understand, human brains have mainly neurons (in a large quantity). As far as we know there are no built in representations. The brain just connects the billions of neurons and from such arrangement consciousness, intelligence, and many other attributes of the human brain are expressed.
What researchers have done are attempts to simulate the behavior of the human brain using a much reduced set of neurons. They started with the perceptron. From then on, due mostly to lack of hardware resources, the achievements are quite impressive and useful, but very far from being how the human brain works.
Research on how the human brain works continues. We need a better understanding and with it we will need inexpensive models that can be used by the billions to simulate a single human brain. At that time, we will have created something that should be able to think and reason as we humans do. We will have achieved singularity.
I am a firm believer that if you cannot explain something, then you do not know the subject well enough. This post is a first entry in a series of articles in which I will cover my understanding on AI (machine learning, neural networks, TensorFlow, Python, DL4J, among other topics). The idea is to start with the current limited understanding of a physical human neuron, how the perceptron attempts to model it and use it with some standard and basic data sets to classify some flowers and hand written numbers. Will follow with more complex models in an attempt to classify and determine more complex tasks.
Please keep in mind that I will be touching of different hardware (e.g., GPUs) and software frameworks and tools (e.g., DL4J, TensorFlow) and publicly available data sets.
Data sets are typically normalized data points of some observation or object (e.g., a face) from which we wish to determine something (e.g., the name of the person). If we wish to experiment with handwritten character digit recognition then EMNIST (https://www.nist.gov/itl/iad/image-group/emnist-dataset) provides us with a set of characters we can use. The idea is to split the set into two groups. One will be used to train our set of neurons and the second will be used to test if our network learned to recognize similar characters. The same holds true if we wish to classify a set of Iris flowers (https://en.wikipedia.org/wiki/Iris_flower_data_set) by using a reduced set of measurements.
Hopefully we will start form the simplest and progress until we get to some interesting projects.
The following figure contains a simple diagram of a neuron.
In the above diagram, the following parts are of interest to our discussion:
| Element | Description |
| Soma | The body of the neuron. |
| Dendrites | Receive signals from the synapses of other neurons. |
| Axon | Fibers extending from the soma of a neuron. |
| Synapses (Axon terminals) | Send signals to the dendrites of other neurons. |
As far as we know, the key elements of a neuron are the dendrites and the synapses. A human brain has approximately 100 billion neurons. Assuming that at some point in time we will understand in detail how neurons work and how they connect, replicating 100 billion (10^11) neurons with the associated connections between dendrites and synapses; which could easily reach 1,000 trillion (10^14) would take a huge amount of computer resources and time to run such a simulation.
Another interesting fact about machine learning, which is a part of AI, uses large sets of data to learn. Let’s contrast it to how a human learns. If we get very close to a hot or cold mass, we immediately try to increase our distance to the object. If we see a moving object coming at us, we quickly move out of its way or attempt to block it. We typically learn these behaviors with very few examples. AI needs dozens to thousands or hundreds of thousands before it gets closer to how a human reacts.
We are amazed at how computers have been able to outperform humans 
playing Go and Jeopardy. We need to understand that such great accomplishments were achieved not by leaving a computer with simulations of 100 billion neurons and 1,000 trillion connections to learn on their own. It took many decades, many scientists, many researchers, and many engineers to develop many technologies that together were able to compete and defeat human champions in games of chess, Go and Jeopardy.
We are putting AI at work in the self driving cars. Such cars, with the use of sensors and cameras are able to build models of the small part of the world which they are interacting with. There a millions of lines of code that are being programmed (coded and tested) to allow such cars to start driving in selected areas in cities and town without the need of a human driver that could take over in case of situations where the computer systems may encounter and would need assistance.
I like to stand on the shoulders of giants and not to assume things. So before finishing this post I watched the following past webinars from the ACM:
| Webinar Title | Speaker | Moderator |
| ImageNet: Where Have We Been? Where Are We Going? | Fei-Fei Li | Rose Paradis (Leidos) |
| AI, People and the Open World | Eric Horvitz (Microsoft Research) | Stephen Ibaraki |
| Using Machine Learning to Study Neural – Representations of Language Meaning | Tom Mitchell (professor at CMU) | Will Tracz (Lockheed Martin) |
All three were very interesting on their respective topics. All of them highly recommended. If you are not a member of the ACM you should consider joining.
Since I started reading, experimenting with AI (e.g., machine learning, deep learning, convolution neural networks) and learning some of their tools (e.g., Python, Pandas) it seems to me that the actual key to be able to generate software that we would properly refer to it as “Artificial Intelligence”, we need to understand how the brain works. Over 50 years ago or so, we started with the artificial neuron. Time has passed and we are still studding and trying to understand how the human brain works. The webinar by Tom Mitchell provides a very good overview as to how we are attempting to understand what goes on in the human brain when it processes natural language using fMRI (functional Magnetic Resonance Imaging) and MEG (Magnetoencephalography).
If you are interested in the state of the art as how we classify words; then “Using Machine Learning to Study Neural – Representations of Language Meaning” and “ImageNet: Where Have We Been? Where Are We Going?” are two webinars you should watch.
I am convinced that AI will get there. I am also convinced that we need to better understand how the human brain works. I agree that for driving a car, flying an airplane, and playing a game of Go is something that a single human being, among other hundreds of thousands of activities is capable of doing. Driving a car or a bicycle is just a subset of the capabilities of a human brain. That is encouraging. As usual, we first need to crawl, then walk and finally run. It seems like at this time, we are in the crawling stage as far as AI is concerned. Not sure how long and how much research will be needed to get to the replication of an artificial brain capable of thinking like we do. I am convinced we will get there.
This set of posts will show some of my attempts to better understand the science / technology / art which is and will continue to permeate most (never say all) of our lives.
If you have comments or questions regarding this post or any other entry in this blog, please do not hesitate and leave me a message. I will replay to it as soon as possible.
Keep on learning, experimenting and coding;
John
www.johncanessa.com
Follow me on Twitter: @john_canessa