Open Government

Chapter 21. Going 2.0: Why OpenSecrets.org Opted for Full Frontal Data Sharing

Sheila Krumholz

The day arrived, rife with anticipation—and laced with trepidation. It was April 14, 2009 when the Center for Responsive Politics (CRP; http://www.opensecrets.org/) made our value-added campaign finance, lobbying, and other political finance data (coded by industry, standardized by organization and individual) fully “open” for the first time in our 26-year history. We had been guarding the “crown jewels”—the detailed transaction-level data that are the building blocks for everything we do—and here we were, financially strapped ourselves, but giving away what we considered precious gems. As an NGO that tracks money in politics, we are passionate about our data and particularly convinced of the need for unbiased, unassailable data of the kind CRP produces. And this data is unique at the federal level. There’s nothing comparable to what we produce, in part because we pour our passion into creating and maintaining this trusted resource. CRP’s data and nonpartisan analysis provide the public with the information they need to hold their government accountable while also educating the public about how money and elite influence so often control who wins political office and what laws get passed.

The Decision to Let Go of the Data

CRP has already had its own share of seriously useful research tools, beginning with our creation of a now well-tested system for categorizing contributions, a creation of former CRP Executive Director Larry Makinson. Larry had actually created this system for state-level contributions in Alaska and came to CRP suggesting that the same system could be applied, and tailored, to federal politicians. This was really the first chapter in CRP’s history. The second chapter began with the launch of our site (then called CRP.org) in 1996, which included political action committee (PAC) profiles, an individual donor search feature, and the essential elements of funding for every member of Congress, including total funding by industry and top organizations. Sections for presidential candidates, congressional committees, and national parties, among others, followed. Then, in 2000 we published online profiles of all political money by industry—including not only totals back to 1989, but also summary highlights such as top contributors and top recipients. Even back then, there were misgivings about “putting it all out there” and how no one would ever call or cite us again because the information is free. Anyone can just take it! Which was, of course, precisely the idea. Nine years later, lo and behold, we’re still standing.

Incorporating data on lobbying—the other side of the coin to campaign contributions—in 1997 was another leap forward in covering the major ways that elite influence can skew our nation’s politics and priorities. The “revolving door,” a companion database covering the traffic between government and “K Street” (shorthand for the lobbying industry), came online in 2006. Keying data from politicians’ convoluted personal financial disclosure reports was another big boon to the study of influence, allowing anyone to more effectively monitor potential conflicts of interest, to instantaneously find those who have both jurisdiction over an industry or company and substantial holdings in it. All three of these data sets were funded substantially by Sunlight Foundation. And since 2003, CRP has offered up in-depth analyses of “heavy hitters,” detailing the biggest firms and specific organizations bankrolling U.S. politics.

And we had already dipped our toe in the “open data” water. First, we had often waived fees for people and projects that had no funding themselves. Even when we did charge—unless the requester was well funded—our fee was usually nominal. But more importantly, in 2007 we began to create customized, private application programming interfaces (APIs), and in the summer of 2008 we published our first public APIs and widgets. (APIs allow users to get a data feed directly into their own sites, providing continually updated, “streaming” data for display on their own sites. Widgets are small graphical presentations of CRP data—and are now available for users to plug into their own sites or blogs.) Users can also “embed this chart” for even easier use of our analyses.

It’s Not Easy Being Open

In fact, during the 2008 presidential election cycle, our team was literally working around the clock to post the latest data following first quarterly—and then monthly—reports. It was insane! As soon as a filing appeared on the Federal Election Commission (FEC) website (see Figure 21-1), we had to gather, process, code, standardize, proof, publish, analyze, and report on these massive sets of contribution data beginning within hours of their being filed.

Figure 21-1. Example of an FEC filing

We could achieve this only because our staff was so dedicated to getting it done (and, let it be said, by occasionally roping in the dean of this research, Larry Makinson). After all of that effort, the last thing we wanted to do was shackle it with subscription fees or further restrict access. We wanted to make it available as quickly as possible to the millions of people who visited our site last year—and the reporters, bloggers, and activists who were waiting to use this information in analyses geared for their audiences.

It also proved a relief, to be honest, to be able to simply put our data out there and say “Come and get it.” This seemed preferable to wasting time creating little data sets here and there for folks who could do it themselves—if only they had access to what we had internally. We were wasting time invoicing and collecting payment. Or trying. (We found we aren’t a very good collection agency.) For years we had tried to scrape together pennies by charging not just for research, but also for the data itself. We almost reached the point of launching a subscription-only website to make transaction-level detail accessible to what we assumed would be “advanced users”—journalists, mostly, and maybe some new media types and for-profit corporations. But it was not a satisfying feeling to cater to these few, relatively well-off interests when people have grown to expect online data to be open and free. It was especially frustrating because our very mission, our core raison d’être, is to disseminate our information to the public, to educate voters, to encourage average Americans to take our research and do something with it.

So, we did it. We chucked the subscription model and instead listened to the steady drumbeat coming from open data devotees (notably CRP board member Ellen Miller, who as cofounder and executive director of Sunlight Foundation has been a major funder of CRP, and Sunlight Labs’ former chief data architect and open data evangelist, Greg Elin), and threw wide the doors. The obvious rationale, as championed for years by our many users—especially journalists, academics, activists, and data geeks—was that our research would be disseminated far more widely if we offered it up freely, rather than selling it by the record. We knew this instinctively; it was never in question. The only question? Cost recovery. And our funders were encouraging us to generate income. For 20 years, we split the difference by charging less than we thought it was worth—and frequently reducing fees, or waiving them altogether.

Finally, though, the positive aspects of open data won over CRP’s board and staff. First, it would reduce those financial and technical barriers that then prevented many organizations from using CRP data to tell the stories of particular relevance to their audiences. So, we were tempted to see what might happen if we expanded our audience to include everybody else’s audience. We were confident we had the best data and we were contextualizing it well, but we wanted to see what others could add to it. We were tired of the administrative aspect of charging—and spending so much time creating custom data sets on demand. Furthermore, in the “age of transparency,” it seemed important to us that we live up to our own standards of disclosure. Sure, we felt justified in retaining control of the value we added to the data, in part because we were funded by private sources, not taxpayers. And those sources were telling us to generate income to diversify our funding, to become more self-reliant. Still, that didn’t appear to make a difference to most of the open data community. And—importantly—those carrots didn’t serve as the only motivators. A stick also appeared: Sunlight Foundation announced that it would only fund projects that made its data wholly and freely available to the public. Losing the support of Sunlight could have been disastrous for the Center.

So, with all of these obvious, even critical, benefits, one might think the choice was a no-brainer. But it wasn’t. At least not from our vantage point.

Creating a New Model for Transparency

In 2008, CRP stood at a crossroads; one we felt might well turn out to be a choice between the yellow brick road and a long walk off a short plank. Our stalwart supporters in the foundation world agreed that our research was as essential as ever—but funding began to contract. The same was true for other organizations in our field, so we didn’t take it personally. “(Foundation) boards get bored,” as one foundation officer—who had funded us for nearly 20 years—put it to me. So, it seemed clear to us that the time for fresh thinking and a new approach was never better. We needed to find a way to attract new attention—and new funding—to sustain the organization and the important research we’ve been conducting for the past 20 years. Like news media, we endeavored to recognize the changing environment in which we’re operating, and then adjust.

Of course, some of this is semantics. While campaign finance may sound like a dated term and has clearly lost some sheen as an issue for foundations, government transparency and accountability has never been hotter. It’s not that the funders thought our work was any less relevant; we just couldn’t call it campaign finance. And it was more than that: lobbyists became the bogeymen of the 2008 presidential campaign, yet perhaps surprisingly, their place in Washington remains salient. Earmarks, both the nutty and those nakedly used as trading chits by members of Congress, still merit front-page treatment—and FBI investigations. Powerful special interests have continued to reap both riches and resentment when energy or health care costs soar, but now they are also standing in line for the Troubled Assets Relief Program (TARP) Recovery Act, and any other public handout they can justify. Since transparency—and tracking elite influence—has been CRP’s essence since day one, this is our sweet spot, and it should be our heyday.

But ironically, despite being a major player in the transparency field since even before the field had a name, that wasn’t our image. We needed to redefine ourselves. As is so often the case in life, image, language, and shameless self-promotion really do make a difference, perhaps nowhere more so than in Washington. Instead of promoting our work, we were slogging it out in the trenches, doing what we do best. We primarily relied on the mainstream press to cite us in their stories for our outreach and promotion strategy. But we slowly began to recognize that this wasn’t leveraging the variety of new methods for disseminating information that had appeared online since the late 1990s. Part of this reality is the nature of our research. As an organization, CRP historically wasn’t looking to be first, necessarily. We were striving to be comprehensive—and right.

To take advantage of the new forms of communicating with the world, we needed to return to our core mission. We generally felt successful in helping to reshape the American public’s distilled understanding of how policy is made (adding an important footnote—“*aided by money for campaign contributions and lobbying”—to that Schoolhouse Rock jingle about how a bill gets signed into law). The next step: continue connecting the dots, telling the stories behind the numbers, and crucially, helping others to tell the stories to their audiences using solid, reliable, CRP-branded data. CRP may or may not get the media citation in the future. (In fact, ironically, if our strategy succeeds we’ll probably get less and less.) But we do hope to be a partner on some groundbreaking investigations by (what’s left of) traditional media, new media, and a whole host of individuals brought in through technology and open data communities. And we hope that all of us, whether together or apart, can create interesting new lines of inquiry and graphical visualizations using this research. Finally, we aim to be the “Intel Inside” for political stories that incorporate facts about how money greases the skids in Washington.

The Future Is Now

So, what’s next? We know what we hope will happen. But really, who knows? It’s an experiment, a leap of faith, perhaps. However, a few things are certain. Data enthusiasts are already happy—and busy—grabbing, mining, and mashing up our data with their own. (Some of them are perhaps happier than they deserve to be—ahem, data brokers. It is a noncommercial license, after all.) Journalists now have an easier time getting the nuggets of information they are looking for—whether simple stats or complicated analyses—without having to call us and explain what they want (previously requiring much back and forth) to put their request “in the queue.” Furthermore, as seasoned journalists take buyouts, our work serving up reliable, value-added data will help fill the growing hole in investigative journalism. We will also provide useful tools at “very affordable rates” (i.e., free) for new journalistic enterprises struggling to compensate for a traditional media in decline.

Activist and nonprofit groups such as Common Cause and LittleSis (a project of the Public Accountability Initiative) are finally able to use our data without worry about cost or delay. And academics may finally obtain all the data, to do the kinds of analyses that only sizable data sets allow without swallowing up meager grant funding. Most importantly, academics will produce scholarly work that never fell within CRP’s mandate or capacity to create, but will expand our knowledge about how money truly influences politics and policy.

Perhaps most exciting is the ideal timing for going open with regard to empowering citizen watchdogs. After the intense interest Americans displayed in the 2008 presidential election and the degree to which people of all partisan stripes now participate and engage in the process and debate, feeding CRP data into the centrifuge of citizen analysis makes perfect sense. We’re confident that data developers and web developers will incorporate CRP information to create new tools for democracy activists, build and empower new communities, and—ultimately—educate and engage more voters. It’s already happening. MAPLight.org is a perfect example, taking CRP data on contributions to members of Congress and mashing it up with voting records. OpenCongress.org uses CRP’s API to stream data into its profiles of bills pending in Congress. Sunlight Foundation will incorporate CRP data into a future “data commons,” which will connect volumes of distinct but related data sets. And new sites are cropping up all the time using clever and useful visualizations, distributed initiatives, and interesting new combinations—usually on low or no budget.

Conclusion

Open data, we believe, will enhance CRP’s reach and utility. Not only will distribution of our research be more efficient and widespread, but we may also focus more of our attention on the critical work of showing the links between money and influence. CRP’s Capital Eye blog and e-newsletter are critical to making these connections for people. But more is needed to explain to a general audience how a tiny elite uses its political influence to shape everything from gas prices to our health care options. And of course, CRP will continue to maintain up-to-date data on OpenSecrets.org because, although bulk data is a huge boon to tech-savvy users, a reporter or blogger looking for one figure does not want to download literally millions of records. Furthermore, most users probably do not need (or possess) the skills or comfort level to accurately process and use massive data sets.

OpenSecrets.org continues to evolve. Concurrent with the “OpenData” release, we expanded our “Member Profiles” to the top 100 organizations and all industries, and simultaneously made these tables downloadable in various formats. We’ll expand these further so that users may ask what should be a simple question: did Congressman A get money from Organization B? Getting that answer is still not as easy as it should be, but it will be much easier once we add all of our standardized employer information (instead of just raw FEC data) to the individual donor search section. This is among the most valuable—and unique—tools that nobody other than CRP offers, yet we hadn’t allowed access to that data on OpenSecrets, even though the donor lookup feature is one of the most popular on the entire website! Therefore, very soon we will reprogram the donor lookup to deliver our clean, uniquely CRP data, instead of holding it back.

CRP is poised to add new features to OpenSecrets.org that maximize user flexibility and to connect the dots more effectively—linking individual and organizational donor dossiers to lobbyists, their clients, and politicians’ personal financial holdings. These will be integrated, and then presented with interactive options, not presented as static add-ons. And they need to pull from useful, related troves of data that other organizations publish and maintain, while also enabling new features on their sites that speak to their audiences. One such CRP mashup compares campaign contributions to earmarks, incorporating data from Taxpayers for Common Sense. And FedSpending.org (and now the government’s own USASpending.gov) makes it easy for us to incorporate government contracts that those earmarks subsidize. Especially as the Obama administration and Congress tackle wide-ranging policy debates such as health care, energy policy, and the re-regulation of Wall Street, these dossiers of leading Washington players could emerge as vital tools to clarify and inform the debate—on our site and, now, on many, many other sites too.

About the Author

Sheila Krumholz is the Center for Responsive Politics’s executive director, serving as the organization’s chief administrator, the liaison to its board and major funders, and its primary spokesperson. Sheila became executive director in December 2006, having served for eight years as CRP’s research director, supervising data analysis for OpenSecrets.org and CRP’s clients. She first joined the CRP staff in 1989 and was assistant editor of the very first edition of Open Secrets, the Center’s flagship publication.