AI: The Importance of Representation

Artificial Intelligence (AI) is a hodgepodge of exciting, creepy, thought-provoking, and incredible awesomeness.
It’s also a mix of representation, inference, and knowledge. Representation involves how an AI agent perceives the environment around it. Sounds and objects are some elements of the environment, and the agent must be able to represent those elements in a way that allows it to understand the meaning. Inference involves actually understanding the representation and, through reasoning and planning, deciding what to do in the environment. Finally, knowledge involves accumulating information by interacting with the environment (much like memory), and also consists of information provided to the agent by the creator.
Wow, that was a lot! We’ll try to unroll inference and knowledge in future posts, but we’ll look into representation today. Let’s start with an example.

Consider the following question: Find the area of the shape formed by connecting (1,2) to (2,4) to (4,3) to (3,1). Looking at the points themselves, it’s difficult to discern the answer immediately, but a good approach to this problem is to visualize it. The points are plotted in the figure below.

Visual representation of points

Notice these points simply form a two-by-two square tilted over, giving an answer of 4. By changing the representation of the points, the answer becomes much more obvious. This is a trivial example, but it hints at something very important — and that is that good representations are really important. This lessons holds in creating a useful AI as it does in our everyday lives. Have you ever been stuck on a problem for a long time, only to find out that the answer can by easily derived by simply approaching the problem from another angle?

Since representation is so important, researchers in AI even have a conference [1] dedicated solely to this topic. They explore ways to convert images, sounds, and words into representations that an AI (a computer) can understand quickly and easily. For example, let’s consider how a black-and-white image could be represented. We could look at each pixel, and convert the color of the pixel to value between 0 and 1, depending on how dark it is. By converting all of the pixels, notice that we effectively convert the image into a matrix! Now this is a representation that an AI can understand.

Simply converting the pixel intensities to numbers works, but by itself, it is not a particularly good representation. This is because the matrix ends up being pretty large for most images, and an AI has a hard time doing inference directly on that. Therefore, an active area [2] of research in computer vision is to compress these images into succinct representations (smaller matrices) in a way that removes useless information and minimizes useful information lost. This way, the AI can do inference on the smaller, more useful representation.

Next Friday, we’ll look what inference consists of, and why good, succinct representations allow for better and faster inference.

Till next time,


[1] International Conference on Learning Representations (

[2] Convolutional Neural Networks are currently the most popular models that learn good representations of images. They are very prevalent in the computer vision community. (









Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s