danielknox.net

My colleague Neil Budde wrote a great post on the Newstogram blog today.

It’s so good I decided to reblog the entire post below, but you should still head over to the Newstogram blog if you want to join the discussion.

We’ve cracked the nut and it’s not even close to 2015

“Delivering news digitally in a personalized manner is a nut many a startup – as well as many established Internet companies and publishers – are desperately trying to crack.”

That’s the way TechCrunch began an article this week about another company entering the personalized news arena. Earlier in the week, we learned that The Washington Post Co. had purchased personalized news venture iCurrent. A week or so earlier, Google News rolled out some modest customization features.

Clearly, the field of personalized news is getting hot. Which makes us glad that DailyMe and our Newstogram platform have a three-year headstart and a unique approach. We’re not desperately trying to crack the nut, we have the nut cracked and are rolling Newstogram out on a range of news and information sites.

Personalization of news has been kicked around for a while. MIT Media Lab founder Nicholas Negroponte described a virtual daily newspaper customized for an individual’s tastes in his 1995 book, “Being Digital.” He called it The Daily Me, a term we later adopted for our company, though he has nothing to do with DailyMe Inc.

In a commentary written for The Wall Street Journal last December, Google CEO Eric Schmidt described a device for news that “knows who I am, what I like, and what I have already read. So while I get all the news and comment, I also see stories tailored for my interests.” In his article, Schmidt imagined such a device being available in 2015.

In our view, 2015 is already here.

Newstogram already is serving up personalized news to users of sites like Variety.com and Impre.com and through several modules on DailyMe.com. Our approach of tracking and analyzing the content users consume and using the resulting individual interest profiles to make recommendations has outperformed other approaches in tests and we continue to improve and refine our algorithms.

We welcome the additional interest in news personalization, because it only serves to highlight our position as the leader in the field.

DailyMe is recruiting for two Data / Research Scientists - role is advertised here on KDNuggets, as well as on LinkedIn.

The Data / Research Scientists will support DailyMe’s Chief Scientist and I as we seek to further improve the already impressive performance of our Newstogram recommendation system.

The Economist had an interesting article this week on the data deluge, in which it argued that, to help users feel like they retain control over their online data, sites need to make more data available to their users:

First, users should be given greater access to and control over the information held about them, including whom it is shared with.

I totally agree that sites should provide greater transparency with respect to tracking and data collection / storage. The Economist highlights Google which allows its users to see what information Google holds about them, and lets them delete search histories or modify the targeting of advertising.

Other sites are increasingly doing this too. For instance, I really like how the Newstogram technology has been implemented on DailyMe.com with a dedicated “My Newstogram” page which shows me what data the site is stored about me, explains how the data will and will not be used, and gives me the ability to correct the data or to opt out of tracking altogether.

Yahoo has similar functionality available through its Ad Interest Manager page (although Yahoo is either tracking a lot less about me or is not as good at determining my interests as they only have me pegged as a generic sports fan).

December is a tough time of the year to get anything done and I now realize that my plan to develop stacked graph visualizations of Newstogram data (which required learning a new programming language) was overly optimistic.

I’ve pushed this out to 2010 but to keep me motivated I have added two stacked graph visualizations to my data wall:

1. Visualization of my Last.fm listening history (full PDF)

2. Visualization of my Twitter stream

The feedback on my tree map visualization was very insightful. Colleagues pointed out that, while interesting, the tree map suffered from the same problem as my other attempts to visualize the Newstogram data: namely it doesn’t address the time dimension. While tree maps and other ‘static’ visualizations (such as bar charts) can display data over a number of different time periods, they don’t really show how the data is changing over time. In the case of many data sets, including the ‘news interest’ data we are tracking through Newstogram, this is the most interesting aspect of the data.

A possible solution is to use a stacked graph visualization. Stacked graphs have been used to visualize a number of data sets including movie revenues, music listening habits, twitter posts, baby names and how people spend their time. So, armed with Lee Byron’s Streamgraph whitepaper, my latest visualization project is to display Newstogram data in a stacked graph.

After seeing Nick Mihailovski’s Google Analytics / Protovis mash-up last week, I couldn’t resist playing around with the Protovis visualization package over the weekend.

My first visualization effort is a treemap showing the popularity of sub-categories within DailyMe.com based on Newstogram data for October 2009 (built upon the Protovis treemap example).



The colors represent primary categories, while the size of each sub-category corresponds to its popularity as measured by the ‘Digital News Affinity’ (DNA) score for October 2009.

The search field at bottom of the treemap highlights certain categories / sub-categories (e.g. searching for “sports” highlights the 14 sports sub-categories).

Check out the working demo (requires a modern browser e.g. Firefox, Safari).

The title isn’t a metaphor for information overload or filter failure… the walls of my office are literally covered in print-outs or drawings of histograms, pie charts, bar charts, treemaps, mindmaps and various other types of data display.

As DailyMe gets access to more and more data through the Newstogram platform, I am becoming increasingly focused on data visualization and specifically how to make our data visually appealing, easy to understand and (most importantly) useful for our clients.

One of my tasks for next week is to check out the open-sourced Protovis SVG Visualization Library and learn how to make my own treemaps (inspired by the team at Google Analytics who just released the video below showing how to create treemap visualizations of data extracted through their APIs) - I foresee more print-outs getting pinned to the walls….

Next New Networks published some interesting data last week about viewership to their family of online video channels, which include Indy Mogul, Barely Digital (home of Obama Girl) and new addition Hungry Nation. The study, conducted with the help of web video measurement firm Visible Measures, showed that the peak period for video viewership was the six hours between 12pm ET to 3pm PT, when many North Americans are presumably looking for a short distraction from work.

[caption id=”attachment_32” align=”alignnone” width=”481” caption=”Source: Visible Measures via Silicon Alley Insider”][/caption]

This trend is hardly surprising given the type of content that Next New Networks specializes in…. short-form entertainment videos. I recall from my time at Channel 4 that short-form videos were popular during the day and long-form videos were popular during the evening (and I bet if you looked at data for Hulu you’d see a similar trend).

By contrast, the general trend for most online news sites is still a morning peak (for instance, DailyMe.com has a readership peak most days between 7am ET and 10am PT). However, I suspect this general trend masks differences between different types of content on online news sites, some of which may provide a similar lunchtime ‘outlet’ to the Next New Networks videos. Newstogram, our soon-to-launch analytics / intelligence platform, will provide an easy way for online news sites to drill down and find the popularity of different categories, topics, people etc. throughout the day in order to identify the types of content where Lunchtime is the new Primetime.

Recently launched site DailyPerfect seems interesting.

Site claims its “innovative personalization technology” is based on “automated semantic analysis”… which sounds a lot like messaging for DailyMe and our Newstogram technology.