Soon, it seems, computers might know our world better than we do. Google is training computers to spot common images on video streams and match user photos of landmarks and other things against its image database. Facial-recognition software could help you identify a complete stranger in the crowd. And now, a group of researchers from Carnegie Mellon University and INRIA/Ecole Normale Supérieure have created an algorithm that can identify cities based on their unique architectural elements and other distinguising characteristics.
The research involved taking 25,000 random samples of photos of cities from Google Street View and letting a machine-learning algorithm figure out how similar features differ in each city. Interestingly, the system seems to be able to indirectly identify cities’ immigration and industrial histories. From their paper:
[I]n Paris, the top-scoring elements zero-in on some of the main features that make Paris look like Paris: doors, balconies, windows with railings, street signs and special Parisian lampposts. It is also interesting to note that, on the whole, the algorithm had more trouble with American cities: it was able to discover only a few geo-informative elements, and some of them turned out to be different brands of cars, road tunnels, etc. This might be explained by the relative lack of stylistic coherence and uniqueness in American cities (with its melting pot of styles and influences), as well as the supreme reign of the automobile on American streets.
The model was, however, able to identify a few defining characteristics in American cities, such as fire escapes in New York and bay windows in San Francisco.
Of course, there’s relatively little utility in just being able to identify images of Paris versus Prague, for example — it’s larger-scale applications that hold all the promise. The researchers identified several in their paper, including figuring out whether certain geospatial factors (e.g., narrow streets) lend themselves to particular architectural practices and giving artists more information to let them draw cities with a high degree of detail without having to study them in person.
Additionally, the researchers write, their work could help create “a stylistic narrativefor a visual experience of a place. … E.g. one could imagine finding a visual appearance ‘trail’ from Greece, through Italy and Spain and into Latin America.”
Or, for the commerce-minded folks, the researchers suggested a few potentially more financially viable examples in the slideshow they presented at this week’s SIGGRAPH conference in Los Angeles. Tuning the algorithm to other objects could help auto manufacturers determine what are the common elements among popular car models, or help consumer-electronics makers (or patent lawyers) identify the unique visual traits that make Apple products stand out.
