State of Tech: Image Corpus Corralling and API Integration

Two of the biggest current tech trends are image corpus ‘corralling’ and API integration. 1 billion images are taken per day and 6 billion photos are shared monthly.

Uploaded photo databases are the new 'data' corpora.

The first moment this shift became clear in Google’s announcement in June 2012 of the ability to recognize images of cats (the most frequently appearing entity in YouTube videos), and in the big data industry’s continual innovation to manage unstructured data like photos. Now, the sheer volume of image-related activity and opportunity for different consumer and commercial applications is making image classification a focal area for the tech industry.

Functionality is being developed for classifying and accessing content – both images and all web content - with tools such as Imagga as a cloud-based image classification program, and OpenCalais, a standard for text-based semantic content analysis and organization. What we might now start to call the corpus characterization ecosystem is expanding into related tools like DocumentCloud that runs uploaded documents through OpenCalais as a service to extract information about the nouns (e.g.; people, places, and organizations) mentioned.

API integration remains an ongoing trend of the year, with integrated API and developer management platforms like Mashery continuing to grow. API integration platforms give companies a means of facilitating and encouraging external developers to access their content to make apps, and give developers a standardized means of accessing large varieties of data content from different sources to integrate in creating a new generation of sophisticated apps and web services. 

Web Analytics: a Precursor to Cognitive Enhancement

Any sufficiently complex, rapidly-responding, and self-adjusting computer system is sometimes heralded as the place that “AI could wake up,” but could be seen more broadly as the venue of algorithm development that might have extensive applicability, including for cognitive enhancement.

Some exemplar complex modern computing systems include big data analytics, high-frequency trading operations, multiplayer video game networks, weather-modeling systems, and now…mobile and web analytics. These latter platforms are becoming increasingly sophisticated, continually updating actual and predicted user behavior data and delivering this information in real-time. Web property owners can watch live engagement with their websites in a heads-up display (HUD) overlay on web pages, as Chartbeat demonstrated at an API-related mini-hackathon at their NYC offices on June 29, 2013.

Some of the most prominent mobile and web analytics companies currently include Google Analytics, Chartbeat, MixPanel, KISSmetrics, Omniture, and Visual Revenue.

The 800 pound gorilla of online metrics is the eCommerce use case, using metrics in endless iterations of A/B testing (e.g.; does version A or B of the website produce more click-throughs and product purchases?) However, it is now increasingly important to measure other kinds of web experiences, for example news consumption where the key metric is engagement time as opposed to a simple link click-through. The type of interaction is also important, where the user spends time on the page, and the type of engagement activity such as reading, commenting, and sharing/referring.

Interconnected Big Data World: API Services Marketplaces

A feature of the data-driven modern world is different companies (e.g.; Walgreens, BestBuy) and web properties (e.g.; eBay) offering access to their data and services via API (application programming interface).

In some part an ingenious crowdsourcing ploy and more broadly having the effect of enmeshing even more deeply the interconnected data world, API access requires a lot of coordination.

A new market segment, API management services, has sprung up to facilitate both public and private (white-label) access. Some of the leading companies in the segment include: Mashape, the cloud API platform, offering developer access to hundreds of APIs, Mashery (acquired by Intel in April 2013), an API network with standardized and richly-documented access to over 50 APIs, and Apigee, a leading provider of API services for enterprises and developers. Other related companies include 3scale, WebServius, and Layer 7 Technologies. A related company Zapier enables task-automation between online services like Salesforce, Basecamp, and Gmail.

API management services are growing marketplaces for the management, distribution, and consumption of APIs. Developers gain the ability to have a standard interface for quickly accessing and working with data from hundreds of sources. Web properties and corporations gain a standardized and outsourced management solution to coordinate external developer interactions and extend their ecosystems.

To Web 2.0 or not to Web 2.0

Yahoo, Google, Microsoft and others are making their APIs open and increasingly easy to use. Often with only a few lines of copied and modified code, a Google map swatch or search bar or Flickr photo cloud providing visual accompaniment or now Yahoo mail information can be added to any website. Google's code is primarily client-side so can run more easily, Yahoo's mainly runs server-side and requires ISPs to have Apache 2.0 and PHP 5 installed.

Those sneaky software companies! At a basic level, open APIs expand the outsourcing trend from user-generated content to user-generated applications too. This actually benefits both the software companies and users.

At the more important conceptual level, open APIs are extending the componentization of software, which has been progressing in fits of cohesion and rollbacks of proprietary standards. As Jaron Lanier and others have long pointed out, the software industry lacks an effective standardized component library and rebuilds the wheel each time. The same point is made in global productivity speak by former McKinsey consultant William Lewis in "the Power of Productivity;" industries that standardized components early became leaders, homebuilding is a notable example.

Users featuring more prominently in the development process as well as other current factors such as Web 2.0 applications being organized in chunkable components may provide a stronger more fungible foundation for software development from which we can all benefit.