Communicat.us

From a simple perspective, "analytics" for the end user means three things: Data + Processing + Display. While purists may say analytics is just the processing, the old adage of "garbage in / garbage out" is apt. So too is the recognition  that great data with great processing can still leave you scratching your head if you can see what is important.

All of this is to say I found it interesting to read that NBC has added a live twitter "pulse" analysis for the Olympics. At the moment I would guess it is must another gimmick to generate buzz. However, there could be real value in what it does.

The purpose of Twitter Pulse is to show the relative volume of tweets and the topics they represent.

Data: The data is Twitter but its not clear how they select tweets about the Olympics vs. all other tweets. For now, we will assume its a good data set.

Processing: Next, there is some type of analysis – presumable to pull out key words such as the sport, the interest, some data keys ("video", "medal", etc.) locations, athletes, and so on. All of this data is processed for one metric, "volume of traffic".

Display: the "volume of traffic" metric controls a box size. The other data keys are used to generate some text overlay and to choose a suitable background image. All of this is then rendered "tree map".

What does it mean ? OR more important to a company, what use is it ?

The resulting data could indicate what people are interested in seeing and thus influence what what content is broadcast as well as what types of advertising will have the highest impact. If this were trusted, then it could impact the cost of advertising as well as the size and demographic of the audience.

However, we need to remember the data set – Twitter. So perhaps this is not a sufficiently accurate sampling to make changes to broadcast content. But, what about using the data to change web content ? It could be a good predictor of web site traffic and thus advertisement selection and placement as well as to focus content.

As an aside, I found it interesting that over the 30 minutes or so of watching the Twitter Pulse, the sport of Curling rarely dropped out of the top 1/3rd of the graph’s boxes.

My observation of "analytics" is that success requires all three disciplines working in concert and more often than not, there are three distinct skill sets – data management, processing, and visualization. If you are interesting in the display component, you might be interesting is looking at "Many Eyes" which not only provides visualization of data set but also lets users share their discoveries – a kind of "social data processing" capability.

Major news outlets are leveraging social media sites for both getting news as it happens out as well as finding the next breaking story. There is both risk and reward to this phenomenon.

Let me preface the rest of this as being a shorter post that planned because I wrote the longer one and my editor crashed (ARGH).

The prompt for this post was the closing credit on the NBC Nightly News podcast. (For the curious, I get most of my news through various internet channels but I still like the capsulated format of the National Evening News. I just don’t like having to drop everything to watch the broadcast at a fixed time.) recently, NBC added a graphic to the end of the news to indicate they are on Twitter, Facebook, and support SMS.

News agencies have adopted the various social media as alternative channels to reach their audience (micro blogging breaking news, streaming video of extended features, etc.). More and more, they are also monitoring these channels for the next big story. The risk to "sourcing" from public social media is "verification". Ideally, all sources are vetted and corroborated, but it’s easy to cut corners in an attempt to scoop the story. It’s risky to get a story wrong. It’s far riskier to make business or security decisions.

There are two common methods of confirming data – authenticated sources (having security credentials and requiring a login to post content) or authoritative sources (building trust over time). Both mechanism have merit. However, controlling the credentials of all users, limits the scope of the user population. This is fine for a private solution but excludes the large data pools generated by public social media services. An alternative is to use analytics on the social media data to create a level of trust …

• how many different sources are reporting the same event
• what medium is used (micro blogging vs video vs SMS)
• what past traffic have the sources reported
• who "follows" the source
• what else is known about the sources
• etc

Monitoring the social media streams and analyzing the content goes far in established a level of credibility in the information. While there is risk in gathering information from unauthenticated sources, there is also great value assuming it is not trusted without verification.

.

While researching sensor swarms, I came across a page over at the University of Notre Dame (EE Dept.) from their Mobile Sensing Systems Laboratory.

The principle is the comparative strength of two signals …

By comparing the received signal strength (RSSI) values in the master and slave nodes (via the left and right antennas respectively), the mobile agent can navigate towards an attractive beacon, or navigate away from a repel beacon.

What struck me is that the general principal was used for aircraft navigation starting in the early 1930’s. Whereas the Low Frequency Radio navigation for aircraft relied on two beacons and one receiver, the RSSI solution relies on one beacon and two receivers.

The preference of one of the other is mostly one of “control”. If you control the beacons and want to simplify the receivers, then a model similar to LFR has advantages. Whereas, if you want to track “others” beacons then the RSSI method has advantage.

For intelligence gathering, it is most likely there is more control over the sensor than the beacon. Further more, when building swarms of sensors, cost, disposability, flexibility (repurposing), and ubiquity are significant factors.

IBM, DARPA, and many of the current catch of reality TV shows are all interested in "stream computing".

DARPA had it’s LifeLog project to "captures, stores, and makes accessible the flow of one person’s experience". Whereas the TV series takes a 3-4 month slice of time for a group of people. The amount of data from these systems is massive and storing and post processing all of it is not really the point, especially when you consider that processing the stream would allow you to "focus" the capture process and make analysis and on-going objective. This is were IBM comes in.

IBM has been working on "stream computing" for the past 5 years. What first caught my attention was the "SmartBay" Environmental Monitoring System Installed in Galway Bay. The project is deploying advanced ocean sensors to collect and transmit real time information.

You can imagine using traditional analytics where data is captured and clustered into data cubes for processing. Once processed and analyzed, there would be a report and a series of recommendations. One of those would be of the form, "If we had had X data, we would have changed Y." this is where stream computing has the advantage.

“The beauty of SmartBay is that, for the first time in history, we can monitor a wide range of ocean conditions on a twenty-four hour basis … This opens up a wide range of possibilities for early warning systems for pollution, the study of fish and shellfish stocks, the prediction of harmful algal blooms, dangerously high waves and even the long term shifts in ocean conditions" – James Ryan

You can read more about Stream Computing here and the SmartBay project here.