January 5, 2016
The Structured Stories project wasn’t intended to last as long as it has. It’s purpose has been to explore whether or not it might be possible to apply computational narrative techniques to general news stories and, if so, to demonstrate that approach publicly. When I began the project, two and a half years ago, I had no idea what would be involved. I had a clear understanding of the devilishly difficult situation facing ‘post-industrial journalism‘, a frustration with the lack of proposed solutions, a background in data modeling and a keen interest in analytical approaches to ‘story’. My goal was to propose and demonstrate an original and plausible alternative to the doomed, article-centric approach to news.
It’s been a long and often challenging road since then. It took almost a year of iteration through data models using a research prototype before a stable representation of structured events and narratives emerged. It took almost another year to build an operational prototype capable of being used by others for reporting into structure and enabling anyone to explore structured narratives. And it’s taken more than six months of reporting, in three separate experiments with a total of 10 reporters, to start to understand the reality of reporting everyday news stories as structured events and structured narratives.
I’ve learned an immense amount along the way, mostly from the many talented and knowledgable people I’ve had the good fortune to meet and work with. I’ve learned how to think about structured journalism and structured narratives, where my approach fits within these fields and why there are particular opportunities from radically changing the paradigm of news. I’ve learned how to talk about the approach, how to work with journalists, and how to navigate the cultural gaps between journalism and technology. Above all I’ve learned that it really is possible for journalists to capture general, ordinary, everyday news as structured events and narratives. The constant theme along this path has been simplification, and I have a growing sense that the approach seems initially complex only because of the need for an unfamiliar perspective.
So what’s left to do? One major task remains – to create a taxonomy of event frames within a significant domain of news. The Structured Stories event frame library already contains almost 600 event frames – abstractions of ‘types’ of news events – and it has become clear that organizing these frames and standardizing their use is critical for applying this approach in real workflows. Building and publishing a taxonomy of event frames for news in a domain – specifically the local and state government domain – will complete the end-to-end demonstration of structured narrative as a viable approach to reporting and publishing news. There are many other tasks remaining. Loose ends abound, as anyone using the Structured Stories platform can see, but fixing those merely requires the time with which to attend to them. Understanding the event frame taxonomy is the last remaining area of exploration necessary before I can confidently propose a design for a real-world product.
October 6, 2015
Focus and engagement are both powerful things. Sometimes it’s necessary to be heads-down on a hard task for a sustained period. Sometimes it’s necessary to engage broadly with others; to communicate, to demonstrate and to collaborate. For Structured Stories the last three months have been about engagement.
The reporting phase of the Structured Stories NYC project completed at the end of July, and the reporting team succeeded in capturing more than 60 stories on local government topics in New York City. As the first independent reporting effort in a structured narrative format, the project produced a lot of findings, which will take time to fully understand. A paper on preliminary results was accepted for the ‘Computation + Journalism 2015’ symposium at the Columbia School of Journalism in New York. The key result is that the approach worked: Ordinary, everyday news stories were reported and published entirely as structured events and narratives by non-specialist journalists. I believe this to be a very big deal.
The next phase of evaluation is now underway, and includes a second, more narrowly focused reporting experiment covering Missouri state government and supported by the Reynolds Journalism Institute at the University of Missouri. The goal of this experiment is to understand how to use Structured Stories to cover long-running state and local government stories, and to build a library of event frames to do so. A separate experiment testing the consumption of news from Structured Stories is planned for late 2015 or early 2016. More on these experiments soon.
Other activities in the third quarter include a visit to NCSU in North Carolina for an invited talk in late July, an ongoing experiment on structuring pursuit stories with a major Los Angeles newsroom, work on a more consumer-friendly user interface with the Reporter’s Lab at Duke. The quarter also saw various articles and podcasts about Structured Stories, an unconference panel at ONA15, several posts on the RJI website, and lots of interaction with a diverse range of people on an even more diverse range of topics. It has been busy, fascinating and fruitful, and I’ve learned much.
But there’s lots to do, and increasingly it will require focus. The Structured Stories platform, which has served well for initial reporting experiments, must be upgraded based on those early findings. Many bugs need fixing, many elements of the API and UI need simplifying, features need to be completed or improved, and workflows need to be streamlined. Also, demonstrating the feasibility of Structured Stories is not the same as demonstrating its usefulness. Finding an initial niche for the technology and exploring various operational and business models will be priorities towards the end of 2015.
Much has been done, but it’s still very early days for Structured Stories and there’s much to do. If you’re interested in getting involved then get in touch.
October 21, 2013
Welcome! This blog is a key component of the StructuredStories project and is the heart of this website. I will use it to share successes, challenges and detours along the path towards narrative-based applications in digital media, as well as posting a few book reviews, technology assessments or commentaries from time to time. Connecting with others with similar interests is much of what makes these endeavors worthwhile, and so I hope that actively and transparently reporting on my progress will help to facilitate that.
I have been keenly interested in computational narrative for several years, and interested in the digitally-driven changes in journalism and media for much longer than that. I began to realize that these interests might overlap, and furthermore that several rapidly maturing technologies might make it possible to realize useful applications within that overlap. My work at Yahoo! took me deeply into digital media use cases and systems (especially automated content understanding and personalization), but I felt that narrative-based approaches to media were as yet too undefined to reasonably pursue within a corporate environment. I left Yahoo! in July 2013 and began to systematically assemble a research and prototyping program aimed at developing proof-of-concept digital media applications based on narratives. As I write this post I am now three months into this program, and my excitement about the possibilities from this approach is growing.
StructuredStories is a company but not yet a startup. This sounds confusing, but is just an expression of how I like to work on complex product design problems. I like to reduce large and gnarly problems to first principles, and then to build new approaches to those problems from the ground up – thinking through, writing down, drawing out, coding up and trying out thoughtful solutions. This approach requires lots and lots of time and intellectual freedom, and is somewhat cross-threaded with the rapid ‘code-test-learn-repeat’ approach to product development that is currently popular. While the risks of ending up in a product ‘dead-end’ are higher with this approach, it also offers the potential for finding the ‘global maximums’ of a product solution space, rather than just the ‘local maximums’ available from more evolutionary approaches. StructuredStories is therefore a vehicle for me to pursue a ‘global maximum’ in digital media products based on narratives, while working in the way that I like to work.
Its hard to imagine anything as simple to the human mind as a story, but applying analytical and computational narrative to digital media is nonetheless insanely complicated. A first challenge is just in thinking clearly about narrative – something that is simultaneously highly abstract but also as familiar as breathing. A second challenge is in integrating what I call the ‘large number of small literatures’ representing the contributions of about a dozen different research fields to the study of narratives in various guises - a challenge that has provided me with several years of enjoyable reading, dot-connecting and occasional but satisfying glimpses of recognition. A third challenge has been developing a working understanding of the foundational issues facing digital media from both the production and consumption sides – issues like the drivers towards media personalization, the changing role of editorial influence in media, the deep causes and effects of habit-driven media and the biases within the existing digital media establishment. A fourth challenge is, of course, understanding a range of new technologies, matching them to the pragmatic requirements of platforms and applications based on structured narratives and then building useful prototypes.
At this stage I make no guarantees about whether this approach to digital media is either technically feasible or commercially viable, however I strongly suspect that it is. Either way I intend to find out, and I invite anyone who is interested in the question to follow this blog and to get in touch.