Australia and Computer Vision

Note: I have more than ever been busy with work and the loss of reach my blog suffered when Google Buzz ended has reduced a bit of the motivation I had earlier when updating it. I won't say I'm making a resolution for a come back this year but I will try to write earlier and write often.

The Three Sisters, a rock formation in the Blue Mountains in New South Wales, Australia.

I was recently lucky enough to attended a big conference in computer vision, the ICCV - International Conference in Computer Vision. It took place in Australia last December and it was refreshing. Australia is a great place to visit but more on that later. One thing that took me by surprise was that I and my collaborators were awarded the prestigious Marr Prize in computer vision. I'm very grateful about that. I won't go into details about our own work that led to this prize in this blog post but I will borrow an excerpt from Tomasz's vision blog to describe it:

"[...]. It is all about entry-level categories - the labels people will use to name an object - which were originally defined and studied by psychologists in the 1980s. In the ICCV paper, the authors study entry-level categories at a large scale and learn the first models for predicting entry-level categories for images. The authors learn mappings between concepts predicted by existing visual recognition systems and entry-level concepts that could be useful for improving human-focused applications such as natural language image description or retrieval. NOTE: If you haven't read Eleanor Rosch's seminal 1978 paper, The Principles of Categorization, do yourself a favor: grab a tall coffee, read it and prepare to be rocked. [...]"

I not only recommend following his blog but also to check out and fund his new venture on Kickstarter.

The conference was a big success and we saw some of the new trends in the field that are getting more established. I will briefly discuss two of those trends: 1) Using geometry for high level computer vision tasks like detection or total scene understanding. 2) Using neural networks and particularly deep architectures. These two separate trends share some history. Geometry was initially taken as the main tool for solving vision problems but time seemed to prove that, at least for object recognition, representing images with 2D templates and using clever machine learning was sufficient to recognize objects. This approach has practically solved the problem of detecting faces. Geometry was to an extent confined to a separate track of classic tasks like reconstruction from multiple images. But in this new trend there is a clear revival for geometry in recognition where people have realized that in order to move forward to make recognition more general we need to incorporate geometry back into the mix.

In the same light neural networks were at some point more or less thought as a general black-box for learning. You show the computer some images and some labels and the neural network learns how to label new images by adjusting weights and filters on the input representations. This weight and filter system has some analogy to how networks of neurons work together, hence the name. They were also used together with 2D templates to learn to detect things like digits or faces and they were successful. Unfortunately there was the problem of scaling to more general neural network architectures with more layers. This required adjusting an enormous amount of parameters that computers were just not able to handle at the time. Machine learning theorists also came up with very solid theory for other alternatives to neural networks like max-margin classifiers and kernels that had better theoretical grounds and comparable performance than the neural networks of the time. And soon people largely stopped using neural networks to attempt computer vision problems and even problems in other fields. This recent revival and interest in neural networks in computer vision is due to recent key successes in training deep neural network architectures that have shown superior performance on several important vision tasks. Today's computers combined with large amounts of data and clever new techniques have all made this possible.

Our paper on entry-level categories doesn't fall in any of the above two categories but instead insists on yet a new focus on vision which is that of finding clever representations of what we should be learning about our visual world. Solving computer vision is not just about translating pixel content into labels but rather about building a higher representation model of the visual world that we can eventually apply to specific instances of this visual world. We have advocated for some time that we can augment this visual world model using knowledge collected from text and studying how people describe images using natural language.

I will take the almost impossible task of using my last paragraph to highlight Australia as a destination. It is an awesome place! Being a huge island, the fauna of Australia is just so different than everywhere else that it feels the closest to a totally different planet. I can only imagine how was the first encounter of non-aborigins of the island with those animals. The kangaroos are like a portrayal of deers taken from a very creative sci-fi movie except they are real. The koalas also have no parallels. The wombats and tasmanian devils are also something to be set apart from the rest of the fauna. I also had a chance to visit Cairns and the The Great Reef Barrier and while this is an awesome view of the underwater fauna I have to say that I enjoyed the above ground fauna the most.

Montauk - Far far away on Long Island

It seems I like pictures of lighthouses. I think I got a better picture here than the one I took of the Fire Island Lighthouse, also on Long Island. I was actually working with pictures of lighthouses some time back on a vision project so I have seen at least in pictures a lot of lighthouses in several shapes and colors.

Montauk is the easternmost part of Long Island and therefore the easternmost part of New York State. Iconic place in the movie Eternal Sunshine of the Spotless Mind, one of the first movies I watched after the Dumb and Dumberer-stage of one of my favorite actors: Jim Carrey. Another serious movie I like from him is the Truman Show, of which I made a reference in my earlier post about mosaics (Starry Night Mosaic Art).

Long Island has many niceties that I haven't yet explored, another movie that is coming this year and portraits some of the forgotten glamour of Long Island is The Great Gatsby starring Leonardo DiCaprio as Mr. Gatsby. I will try to visit a couple of those mansions from the 1920's before the end of the year and post some more pictures.

2300 Jackson Street

The King of Pop, the Greatest Entertainer of All Time, I don't think any superlative can account for the genius of this man born in Gary, Indiana, a small city a few miles away south of Chicago. The neighborhood doesn't look in any way special, it looks just like any neighborhood in a suburb. The town is surrounded by big factories and it doesn't feel like you are getting closer to this seemingly forgotten landmark. It feels like there is usually nobody around but it only takes a minute or two to start seeing cars stopping and fans getting out to take pictures. This is not the Neverland ranch or the Hollywood Walk of Fame, this is the true place where you can relate to Michael Joseph Jackson, the child that grew onto one of the biggest talents humanity has witnessed.

At the intersection of Jackson Street and Jackson Family Boulevard.
Gary, Indiana.

Whiteboard outside the house. Lots of messages already.

Going South

This is an East Coast story: Finally I found some time after a couple of conference deadlines to embark on a road trip from Long Island to Virginia Beach. Stops at Jersey City (NJ), New York City (NY), Newark (NJ), Hamburg (PA), Hightstown (NJ), passing through corn fields in Delaware (DE), Pocomoke City (MD), Virginia Beach (VA), Richmond (VA), Baltimore (MD), Atlantic City (NJ), Jersey City (NJ), New York City (NY) and back to homely Port Jefferson Station (NY).


View East Coast Trip 2012 in a larger map

Old Google maps still a wonderful tool for displaying geographical stories. 

The Fire Island Lighthouse

Fire Island Lighthouse

Fire Island is a smaller island connected to the south shore of Long Island. I have been there a couple of times already but so far I have not been able to get there before 3pm so that I can get into the top of the lighthouse. I have not been able to make it even before 4pm so that I can make it to the museum that is located in the building besides the lighthouse. Still there are some nice afternoon views and you can more than often get to spot deer. At least to some of my friends this seems to have become  the place for when you are getting bored at home and feel like driving.

Interesting fact is that Fire Island is one of the islands that are part of the Long Island and New York City barrier islands. Another of these islands being Coney Island for which I also dedicated a blog post some time back "The New York Aquarium and Coney Island". I will update this entry once I get to enter the lighthouse and the museum (and this might not happen for quite a while), for now I just wanted to share the picture I took with my fancy smartphone.

San Francisco Bay Area - Summer 2011

Golden Gate Bridge insider's view

I'm writing this after a long gap since my last post due to the burden of moving all the way to the west coast and signing contracts for every other thing imaginable that comes with relocation. I'm working in multimedia content analysis for Android during the summer. Probably the couple of friends who read my blog already know what is Android but I will anyways write here the explanation I gave to my parents: Android is an operating system that can be installed in a variety of smart phones manufactured by different companies. It's pretty much what Windows used to be for PC's. Note: Maybe you already noticed, but obviously this doesn't represent the views from my employer in any way! The thing is I didn't feel like explaining Android is open source which means you can download, read and analyze its source code and it has a very open model for writing and publishing new applications, including games!

Yoda fountain outside of Lucasfilms

Regarding the San Francisco Bay area, there are a couple of new places I visited this time, from the Santa Cruz Beach Boardwalk to the south up to Sausalito to the north. Sausalito is on the other side of the Golden Gate bridge and I include a picture here while crossing it. Another iconic place I wanted to visit for so long was the Yoda fountain outside of Lucasfilms. I felt like the security guard outside the building gave me the "oh here it comes another freak"-kind of look :) and then gently pointed that we can park anywhere for 20 minutes, so I guess lots of Star Wars fans go there just to take a picture of the fountain. Next to the place is the Palace of Fine Arts Theater and the Exploratorium. So if you visit any of those places then you might want to take a walk and take a picture of the iconic Yoda fountain.

Washington DC, Philadelphia and Baltimore

At the beginning of this year I was part of a two-day trip crossing through five states in the East Coast of the United States: New York, New Jersey, Pennsylvania, Delaware, Maryland. Following the same procedures as in my Niagara trip we embarked on a shuttle at Chinatown. Here are some pictures we took along the way:

US Capitol in Washington D.C.	Independence Hall - Philadelphia, Pennsylvania
Baltimore Inner Harbor, Maryland	Potomac River - Washington D.C.

The Museum of Modern Art - MoMA in New York City

Me and a group of friends visited MoMA in New York last November. Being part of the State University of New York system grants you some benefits on some of these venues and visiting this important museum in New York is not one we wanted not to take advantage of. The streets of the city were particularly crowded since it was Thanksgiving season and people were buying and selling stuff more than ever.

I had already visited more than two years ago MoMA San Francisco and admired some of the most relevant pieces in contemporary art. San Francisco and New York are very different in several respects, but they are definitely the two most important cities in the US when it comes to art. I used to do painting and charcoal drawing myself before going to college. I attended a couple of academies when I was a kid but when deciding my career I didn't pursue this further. Nevertheless, this helped me gaining some knowledge and appreciation for art.

I noticed that one of the pieces that most attracted the attention from the public was Starry Night by Vincent Van Gogh. But I had the chance to see other works that I had only seen in the books from my art classes from when I was a kid.  Les Demoiselles d'Avignon or the Three Musicians from Pablo Picasso are a good example. In the picture in this entry I am with a Jackson Pollock painting, although his art of composition of simple forms in complicated ways has received mixed critics, one of his paintings is considered the world's most expensive painting, sold in 2006 in 140 million dollars. The art of Monet, Mondrian, Dalí, Miró, Warhol also made the trip to the city more than worth it. Next time I will try to visit the Metropolitan Museum, another big one for art in New York City.

Machine Learning and The New York Academy of Sciences

Last Friday some friends and I attended the 5th Machine Learning Symposium organized by The New York Academy of Sciences in Manhattan. People from institutions nearby presented some of their research going in Machine Learning and Learning Theory. Even when I have been mostly working on Computer Vision and Images, the kind of projects that I am working on have a lot to do with Machine Learning so I'm very excited that there exists such a large local community interested in those topics in which I mostly rely upon and might potentially contribute. I would like to comment some of my impressions about the Machine Learning and Computer Vision community ecosystem and how there is an EM-like process going between these two.

Sometimes people in Computer Vision contribute to Machine Learning and sometimes people in Machine Learning contribute to Computer Vision as well. Computer Vision scientists get better results as they get better understanding of the human visual system and also when they get better ways to learn from data. So they often find themselves coming up with new techniques to discover patterns on image data. On the other hand Machine Learning researchers will often try

Charles Darwin at The New York Academy of Sciences

to prove their theories on interesting datasets. Interesting could mean complicated but could also mean datasets that are likely to be reproduced by a natural or artificial process. In this case image statistics are a very interesting case because they come with their own encoded statistics, the so-called natural image statistics. Another application that comes with natural statistics is NLP or Natural Language Processing. So very often Machine Learning researchers find themselves trying to discover patterns on either images or language. And maybe I should also mention that there are those who these days are trying to do them all three and I happen to be a member of such groups which is good because it lets me see closer what's going on in each of those fields. The symposium was very well balanced in this regard, it presented projects dealing with images, text and pure learning theory. I'm looking forward to attend the meeting next year and perhaps presenting some of my own this time.

I also want to stress how impressive New York looks from the New York Academy of Sciences auditorium. The whole magnificence of the giant skyscrapers feels like it can almost be touched and sensation of flying is astounding. You can see in the picture here the shadow projected by the building I was in onto the other buildings. Also I was very happy to find the bust of Charles Darwin in the inside. The eminent scientist that one day decided to pursue his research in the Galapagos Islands in Ecuador in his quest for uncovering the mysteries of evolution.

View from The New York Academy of Sciences

Niagara Falls

With summer about coming to an end and research starting to get interesting I embarked on a trip to the north-est part of New York to meet the sublime Niagara Falls and the Maid of the Mist experience. I include in this post one picture of Niagara Falls at night with Canada on the other side. 

If you're thinking about going to Niagara Falls from New York City you will probably search for tours that take you there on Google. Independently of what website you find, you will probably end up taking a bus in China Town and paying some extra money for going to either Corning Museum of Glass or Thousand Islands or both. We chose the Thousand Islands combo. Thousand Islands is an archipelago in the border between Canada and US, which for interesting that it might be you will hardly find it very interesting after visiting the majestic Niagara Falls with the Maid of the Mist experience. That said, the Maid of the Mist experience is a short boat trip that takes you as close as you can get to the Falls, so that you can immerse yourself in the mist created by the drops of water in the air. The trip was an experience on itself also, people from China, India, Latin America, Europe made part of this multi-lingual speaking bus. And we quickly passed through a number of cities in north New York, namely Rochester, Buffalo and Syracuse. 

Summer time is over and my first year of graduate school is coming to an end. Looking forward to what comes next.

Manhattan Pier 17

The night before taking my flight to California I spend some time with friends at Pier 17 in Manhattan, a place from where you can have a view of the main bridges of New York, more prominently Brooklyn Bridge. I include here a picture we took that night.

We ended up in one of the student housing apartments of Columbia University where one of our lab mates is spending the summer. Finally, I would also like to use this short blog post to recommend the blogs of my friends. I include here the links:  Life Is But A Dream and Oh, by the Way.
Which reminds me that I might need to choose a fancier name for my own blog. 

The American Museum of Natural History

One of the largest museums in the world, the American Museum of Natural History brings a world of knowledge and visual information like no other place. The feeling that accompanied me along my visit to this vast sanctuary of science was that of a grown up who would have wished to visit this place when he was a little kid. The ancient civilizations, the jungle animals, the stars, the dinosaurs!

This is the way I started 2010, by visiting this iconic place in New York City which is located besides Central Park. I went there with a friend and it took us an entire day to walk around the whole place and read some of the captions about species, places and points in natural history. This is one of the first museums I have visited in the East Coast of the United States and the largest I have ever visited. I can't wait to see the other large museums of New York, my next planned destination is the Museum of Modern Art (MoMA).

The New York Aquarium and Coney Island

Coney Island in the south coast of Brooklyn is an interesting place. According to Wikipedia, it used to be an island and there's a lot of history behind the name But what I could notice when I was there about what is said in Wikipedia is the fact that this used to be a place for amusement parks. They say that before World War II this was the main destination in the US if you wanted to go to amusement parks with roller coasters and similar kind of rides. After some years of neglect it lost its popularity and even though when there are still some parks and rides there, it's not the top place anymore. I could personally see one of the old wooden roller coasters (look at the first picture) beside the New York Aquarium, which is the actual place that moved me and a friend to visit Coney Island in southern Brooklyn last year.

The New York Aquarium is yet another of those places you wish you had visited when you were of a younger age. Lots of big aquatic animals that you can see as close to you as some few centimeters. I could see for the first time many animals that I had only seen before through the eyes of Jacques Costeau when I was a child and used to watch his journeys on TV. At the end of the day we walked through the empty beach of this in another time acclaimed iconic place named Coney Island.

Update (Apr 19, 2011): I visited Coney Island again last Sunday and this time under some peer pressure I had a ride in the Cyclone, the wooden roller coaster that can be seen in the first image in this post. I can say it is a great experience to ride this engineering monument that dates back to 1927!

Summer Time - California and Silicon Valley

Arriving to San Francisco International Airport doesn't really show San Francisco, from there you have to take a BART train in order to get to the downtown. The feeling that you get after getting off from one of the subway stations and you first see the real San Francisco can hardly be described, so I took this picture right after getting there:

Yes, at the beginning of this summer I traveled to California. I attended CVPR 2010, one of the main conferences in Computer Vision and Pattern Recognition. The world of datasets keeps growing and getting richer, most notably imagenet and the scene understanding database.  I could also see the demos and the insightful research that's going on in every major research center. This was a really insightful experience that I will try to use in the best way I can for my own research projects.

Going back to the San Francisco Bay Area and Silicon Valley brought back many good memories. I visited the headquarters of AMD in Sunnyvale where one of my classmates is working now for the summer. Since they bought ATI, they have been heavily involved with graphics and this is reflected in many of the details of their facilities although overall still keeps the feeling of a traditional software company. In contrast, I then visited Google and for my surprise lots of people outside of my old group could still recognize me. I had lunch there and then spent some time playing games. By night I and some friends visited Castro Street which looked more crowded than it used to be, night life in Mountain View is apparently getting better.

Right after my trip to California I shortly visited my home country. This was probably the last time I could see my sister in the following two years since she's planning to move to France to get a masters' degree in Biology. I had a very good time there although I didn't have the chance to visit many places. But as I have always thought, I prefer the people over the places. Now I'm back to research and work at Stony Brook, which is also good.