Victor Brown and I lead a workshop session today on “Human Interaction – How Do People Play in SOA?” Over the next couple of days, we’ll divide up the slides into vignette and share them here.
One vignette was the value evolution of feeds and their value to intelligence gathering. Here is the text from that section of today’s workshop …
The evolution of “Web 2.0 Man”
cartoon credit: Noah Kroese, twoplayer comics
In the beginning, John used a feed reader and only listed blogs he had found from work or friends – most were industrial (every post had a narrow theme); a hand full were by people he has met in his work over the years and so he was interested in both the primary theme as well as the random personal posts; and aa few were family and friends and they use their blogs as “new letters”.
Later, John found other sources with feeds – a couple of forums; a wiki that related to his project at work; and the company’s on-line file store supports feeds – he could subscribe to documents updates, new documents by a given author, or documents based on tagging.
Recently, John had felt like there was too much “noise” in his feeds so he tracked down and inserted a feed aggregation service – three of the industrial blogs had expanded to technology which was not relevant to his work; a few of his contacts had taken to blogging more about family; and two members of his family were getting more political in some of their posts. John used the aggregation service to apply filters to feeds so only matching material passed through to his reader. This cut down on a lot of the “noise”. John also realized the service let him subscribe to many more blogs and other sources that he had ignored in the past because they had very few relevant posts.
Now, John has learned he can use the aggregation service to find content without knowing the original feeds. When John is assigned to a new global incident (such as a natural disaster or humanitarial project) he immediately creates a new grouping with the aggregation service, giving it a combination of search terms, names of experts, geographic details, and a time window. John has also started to use the tagging and ranking features on the aggregation results so the system refines the feed for John.
In the future, John wants the creation of the new aggregation group to be automated through the service’s REST interface and be triggered anytime a new incident occurs. He wants to receive a notification of the new feed and he wants it to also go to everyone assigned to the incident. John’s teams have become accustom to the feed information and with new social communities features, team members rank feed results so the group gets better data and can share the research work. John would use the new feed report capability which shows which feed results were ranked highest by his team’s members.
Conclusion
A. For this to work, the content systems must support feeds. Most of sources in this scenario were already enabled for content feeds but historical content and systems often are not. In the above scenario, the team spends only 20% of their time processing all of the external data but 80% of their time in manual processes finding and disseminating the internal content.
B. The feed aggregation services feeds to be filtered to just the relevant content and as such, many more feeds may be used as source material without reaching information overload from extraneous content. Because feed aggregation uses data from the union of all subscribed feeds, a “grouping” will return results from feeds the subscriber may not even know exist. This exposed the consumer not only to sources they had not yet discovered but also to option to further investigate those new sources.
C. In this scenario, the feed aggregation service was “out in the internet” which means it could not see back across the organization’s firewalls to internal content. The solution is to deploy and enterprise wide feed aggregator which sees internal content and can also reach out through the firewall to external sources.. As with most social technology, the value of the aggregation service increases with the number of users and the volume of content it processes. An incremental approach is to combine an internet based service with and enterprise deployed service to improve results – for each grouping created on the enterprise service, a compliment could be generated on an external service with the output feed being consumed by the enterprise service.
The new class of feed aggregation – where feeds are filtered, feed content is accessible without the explicit subscription, and where algorithms can affect the feed results is often referred to as “Enterprise RSS”. Research is working on improving the algorithms with the hope that feed aggregators may be a central component to intelligence gathering including finite duration exercises such as emergency response, global events, and humanitarian missions.
A data mashup is one where many different data sources are brought together into a single visualization to speed the user’s ability to understand and process the relationships between data and form answers and take action. Popular examples of data mashups include examples in real estate and city planning as well as more targeted mashups like flight planning for pilots. The majority of data mashups are created by programmers using the published interfaces provided by data sources and often uses a public display solution like a map with a published interface. 
