Robert Townsend’s article about the Future of Peer Review on the American Historical Association’s web site is well worth reading for its thoughtful analysis of both the challenges of traditional peer review in the digital context, as well as some goals for perfecting the system.

It’s a long, fairly grueling process to get a peer-reviewed article published in a scholarly journal, even in a predominantly e-journal. I’ve been thinking about this as I completed the process for my article, The Progression of Digital Publishing: Innovation and the E-volution of E-books. The article touches on peer review in the context of an ongoing, digital resource and regarding digital textbooks, but it’s not the subject of the article, which focuses on innovative electronic texts. At the end of the publication cycle, however, I started to wonder about the balance between the rigor of the process and its pace. I wrote the article in November 2009, expanding on presentations I’d given in October. After review, revisions, and approval, I submitted it to the International Journal of the Book in mid-January, whence it journeyed merrily through double-blind peer review (contributors also volunteer to be a reviewer for the journal), a bit of revision and updating in April (Apple’s iPad was now a reality instead of a rumor), typesetting, and finally, in June, publication. Toward the end, I must confess, I was a bit winded, wondering if proceedings would ever come to an end (having worked with hundreds of authors over the years, I know I’m not alone in this feeling). Seven months, more or less, which is fairly rapid for a journal, so I’m not complaining.

This article isn’t about a new scientific breakthrough or method, obviously, or a gathering and manipulation of data, it’s nothing particularly controversial. I am not a professor, so the self-interest of tenure, which may steer some scholars onto the peer-review highway, is not at all applicable in my case. Both this article and my previous article on the subject were conceived and executed primarily for intellectual exercise. Peer review, one expects, contributes to accuracy, rigor, and, I suppose, legitimacy, and these motivated this scholarly pursuit.

Michael S. Gazzaniga, in Human: The Science Behind What Makes Us Unique, says “to separate the verifiable from the nonverifiable is a conscious, tedious process that most people are unwilling or unable to do. It takes energy and perseverance and training. It can be counterintuitive. It is called analytical thinking. It is not common and is difficult to do. It can even be expensive. It is what science is all about. It is uniquely human.” (p. 273-274).

Peer review is not flawless, nor is research. For an example, Jim Lindgren’s Yale Law Review article, Fall from Grace: Arming America and the Bellesiles Scandal, gives an often quite fascinating account of the twists and turns of the academic scandal involving Michael A. Bellesiles’ book on the history of gun control in America. Bellesiles’ book was awarded the prestigious Bancroft Prize, subsequently rescinded, for the first time in the award’s history, after flaws, and perhaps even fraud, in the author’s methodology came to light. Lindgren mentions that the entire scandal may have been avoided through better editing (and presumably better peer review) at the Journal of American History, which published Bellesiles’ original article on the subject. The book was published by Knopf, which while assuredly not an academic publisher, one would expect to nevertheless have an interest in publishing accurate books.

A paper by the President of the RAND Corporation, James A. Thomson, about the increasing polarization in our society, goes beyond peer review to the concept of bulletproofing: “By far the most important is the quality assurance (QA) process, especially the concept of ‘bulletproofing.’ To some of us who were trained to believe that the most important part of the QA process is the scientific peer review, this can sometimes be an alien concept. Of course, the scientific peer review is the sine qua non; the science must speak. But if controversy lurks, bulletproofing is essential. This involves thinking in advance about the political lines of attack against the results and then identifying individuals who might come from those political quarters. Such individuals should be brought into the review process.”

Dan Cohen, in a blog post about the social contract of scholarly publishing, posits that an exchange of analysis for attention, as it were, is central to academic value and reward. He underscores how the supply side of this social contract—writing, peer review, editing, publishing—and the demand side—the space for attention and consumption—must be aligned, and wonders if these may be slipping out of gear in the digital age.

What about a Threadless-style peer-review system—treating academic articles like crowd-sourced designed, limited edition t-shirts? (Threadless, in my opinion, has produced one of the web’s most successful and innovative mashups of creativity, social media, and e-commerce.) Is an e-book model a la Smashwords feasible for academic monographs and articles?

There are already numerous examples. Rice University’s Connexions, for example, is a digital textbook platform that accepts everything. Their system of peer review is the philosophy that the cream rises to the top, that better modules will be rated higher and therefore get more use, a self-reinforcing system. MERLOT (Multimedia Educational Resource for Learning and Online Teaching), is a community designed to share peer-reviewed online teaching and learning materials.

Brian Whitworth and Rob Friedman lay out in a First Monday article the tenents of an online Knowledge Exchange System that could reinvent academic publishing with a system that accepts all, reviews all, and publishes all. This system would allow for both anonymous expert ratings as well as general reader ratings in order to increase both dissemination and discrimination, both rigor and relevance. User ratings could be broken down by criteria like relevance, rigor, writing, comprehensiveness, logical flow and originality.

These new systems of scholarly publication, peer review, and dissemination won’t come easily, necessarily—the central question remains as always “who pays for this?” not to mention “who has the time for this?” Yet a hallmark of this digital age is experimentation, exploration, and upheaval.

We live in interesting times.

Saving scholarly publishing, and saving civilization.

In 16 minutes.

The full text, and the YouTube videos, are at:

http://www.nap.edu/staff/mjensen/scarcity.html

or you can watch Part I (missing my preface, that’s available in the full text):

and Part II:

A few segments from the text:

The realities I see ahead of us, in the next ten to fifteen years, militate for some radical strategic choices, in the next three years.

I believe that we must shift our business models — publicly, transparently, intentionally, thoughtfully, but radically — to a digital one, with open access as the backbone of scholarly publishing. We must do this to survive a tremendously turbulent next decade, and to ensure that our mission, and its survival, continues to be fulfilled.

…

But CO2 does something much worse. While we bicker with global-warming deniers, the ocean is getting more acidic. Excess CO2 plus ocean produces carbonic acid. Ocean acidification is a clear and present danger. A slight rise in acidity dramatically affects calcium-carbonate-based lifeforms, like most plankton, shellfish, and coral, the cornerstones of the ocean biosphere.

If humans do not drastically reduce our CO2 output in the next ten years, our rich, biodiverse ocean will become an acidic, jellyfish- and algae-filled cesspool, in our lifetimes.

If, over the next decade, humans continue doing what we have done for the last fifty years, then we will construct our own hell, and our grandchildren will curse our names.

…

Within the context of a world in crisis, we *must* demonstrate that we’re radically rethinking our relationship to the future. We must demonstrate that we are part of the solution, not part of the problem. We must seize initiative now, and start making changes as fast as we can.

Open access + digital publishing will help get us to a sustainable world, and keep us in the mix.

Imagine, in five years, a different income stream where 50% of your income comes from some kind of value-added digital sales, and 25% from print-on-demand, and 25% through institutional support of fixed costs. Dissemination and societal impact will increase 50x, because the material is openly available and promoted online.

With that kind of documented dissemination of scholarly value and University brand, to the broadest public, no dean would be motivated to cut the support that enables scholarship to thrive online. And, our CO2 production will be radically decreased.

The presentation was controversial, and raised both some hackles and some hair-on-the-back-of-the-neck. Far more congratulated me than condemned my analysis — and many said they were rethinking strategy in light of what I showed them.

It was risky, but knowing what I’ve learned over the last two years doing the Apocadocs project, it was a risk I needed to take. Time’s a-wasting.

I’ve been interested to see the  responses, and this post can become a response locale — I’m linking back here from the fulltext, in hopes that some discussion can ensue.

Time past long past, in the mid-90s, I held the lucky position of being the first electronic publisher at a major university press, holding the unlucky responsibility of bridging the divide between the digitizing journals division, and the not-yet-digital books division.

While not as extreme as Snow’s “two cultures” of sciences and humanities, the distinctions between the two cultures of books and journals became clear, as I learned the two enterprises. (Sometime later I’ll address the distinctions between library culture vs. publisher culture, and between technologist culture vs. library vs. publisher culture).

What I learned, in short, and necessarily bluntly:

Journals are about throughput. Books are about craftsmanship.

This is not to demean either publishing variant — they both serve key scholarly needs. But in much of the discussions on these topics, too often “open access” is thought to mean the same thing for every kind of document. Non-publishers in particular often presume that the same rules apply to encyclopedias as apply to monographs, as apply to journal articles. But in publishing, at least, it’s not the case.

Every book is unique. At that time they were each treated uniquely: in editorial treatment, in type and cover design, in marketing plan, in discount schedule, in presumed audience. Each book was a child, nurtured in its embryonic and infant stages, eventually dressed up really nicely as a toddler, and sent out into the world once grown-up.

The acquiring editor had a vested parental interest in ensuring that this special, wonderful thing would get the life it deserved. He or she pressured the Marketing department for appropriate promotion. She or he pressured Production to make sure it was designed appropriately for the content. It was a unique, special, rich, complex, discipline-affecting work of staggering value, at least within a tiny slice of academia. The editors were proud to have acquired it. They wanted to reach the people who would be moved by it. As a consequence, each book was a polished gem.

Journals, however, were all about throughput, driven by the schedule of subscription: every quarter, or every month, the articles were bullied out of editors, who bullied their writers and their reviewers for material. And the articles came through the pipe.

Each article needed to be fit into the specific journal’s style, look, and feel, and turned into something that could be distributed to subscribers and libraries. There are no “acquiring editors” on most small-market, scholarly journals. Each article is one of ten, or twenty, in the issue. It’s not an act of craft — it’s an act of meta-craft. Each article is a small part of what will, overall, improve or sustain the value of the journal.

These two cultures may explain why, at least to my mind, journals were among the first to “go digital.” It made eminent sense: it’s easier to “digitize” the throughput process than the nurturance process.

Journals were already template-based (essentially, CSS-ready). Journals publishers were already greatly focused on automating processes. Economies of scale could make journals production and throughput much more efficient. It also meant that the throughput process could be undertaken by libraries, using smart software (as seen by how many libraries are becoming “journals publishers”).

But I’m not so sure that template-based automated publishing makes sense for book-length scholarly monographs, and other book-length works. Perhaps that’s one of the reasons that e-book standards have been so slow to catch on, in the book publishing industry.

We book publishers each think that we know what is right for “our kind of publication” — each of which is unique, special, and unmatched. We want what is right, in terms of marketing, and promotion, and audience, and significance, for that special child.

Is that so wrong? I don’t think so — in fact, I think it’s a powerful strength we don’t want to lose, as we march toward digital universality.

But it’s also a powerful constraint.

Libraries, who are also about throughput (and organization) of products, are an ideal partner for journals production. I’m not so sanguine about libraries being ideal partners for production of the “special.” They didn’t evolve that way, nor are they optimally staffed for the sort of specific promotion, marketing, and outreach required for each unique book-length publication.

How publishers make things “open access” depends on technical infrastructure, and publication content types, and available skill sets and online savvy. But it also depends on the nature of the product.

That every book-length monograph deserves special treatment is open for debate; that some of them do, is beyond discussion. How our academic culture makes that happen — how the funding, the staffing, the nurturance is enacted — may depend on how we understand the processes and purposes of publishing.

I want to first talk about a surprisingly little-known phenomenon in Eastern Europe, that I observed in the years immediately following the fall of the Soviet Empire.

Between 1990 and 1994, I worked a lot with scholarly publishers across Eastern Europe, including some seminars organized by Peter Kaufman, from whom you’ll hear later. My deepest experience was in Czechoslovakia, before it split.

The Soviet scholarly publishing model was to have every university publishing its own stuff–introductory biology coursebooks, collections of essays, lecture notes, monographs, research–at the professor’s behest. There was heavy subvention from the universities, which were of course subvened by the state. There was no economic feedback system, because there were no cost recovery systems beyond a token fee of a quarter or so per “scripta,” as the class publications were called. Traditional hardbacks cost the equivalent of a pack of cheap Czech cigarettes.

Editorial selection scarcely mattered. Every year, a few works were designated as worthy of being put in hardback, usually in an attempt to give their universities a medium of exchange — trading books for books from the outside, non-Soviet world.

The overall system separated to the point of immeasurability, by massive bureaucracy, all publishing costs, and in fact discouraged cost containment systems based on merit or audience. Instead, decisions were [often] based on old-boy status. By having the right connections, an important professor could arrange to have 50,000 copies of his book on the aerodynamics of bat wings printed, in Czech; this was to his advantage, because his royalty was based on numbers printed, rather than numbers sold.

Other publishing houses published scholarly work in philosophy, science, metaphysics, etc.; they had somewhat more freedom of choice of what to publish, but their work was also heavily subsidized by the government, and even in smaller specialty houses, prices and print runs were at the whim of, ahem, important people.

When the Soviet Union collapsed, there were warehouses with hundreds of thousands of copies of the writings of Stalin which nobody would ever buy. And the warehouses also held about 49,800 copies of that bat wing book.

In the prerevolution days, a socially supported, exceedingly expensive publishing industry created very inexpensive books, and that deeply affected the Czech culture. New books came out once a week, and the bookstores were like flowers in a field — every square had bookstores, every tram stop had a cardtable selling books. When I first spent time in Prague, about nine months after the Velvet Revolution, I saw everyone — and I mean the butcher and the hardhat and the professors alike — reading on the trams, the metro, the streetcorners, and lining up to pay a few crowns for new titles — and not escapist trash, but history, philosophy, science, metaphysics, in Czech. This system of subvention created a highly literate, well-educated populace, who read ideas for fun, all the time.

In the aftermath of the collapse of the Soviet centralized economy, suddenly universities, whose subventions were being completely reconsidered by new governments, were telling their “presses” that they had to become self-sufficient in two years, and many were told they had to start giving money back to their universities. For most of the Czech university policymakers, their naive recipe for capitalism was a pinch of slogan-level ideas picked up from Dallas reruns and the Voice of America, a dash of Hayek, then spiced with understanding gleaned from dinnertable conversations over the years.

Their consequent policies had no consideration of the realities of publishing costs and cost recovery, no understanding of the infrastructure (like editorial selection, distribution, and warehousing, not to mention computers, databases, predictive knowledge of the market) required to have a viable publishing marketplace, no understanding of the place of scholarly publishing in the educational system, and no recognition that in a revolutionary economy, nobody would have spare money to make discretionary purchases.

Four years after the revolution, the prices for books had become ten to fifty times as expensive as they used to be. The publishers who were surviving were subvening their own translations of Derrida by publishing – literally – soft-core pornography.

Bookstores closed down everywhere. Publishers closed down everywhere. And people stopped reading every day. By 1995, nobody was reading metaphyics on the tram. A quarter of the university presses I knew of were closed, over half of the small scholarly publishers I’d known, well over half of the bookstores I knew of in Prague were closed, and the scholars I’d befriended were telling me that they couldn’t get anything published anymore — there were fewer outlets than ever.

Neither model was right–the absurd redundancies and inefficiencies of the Soviet system were far too costly, but its result was a marvelously high level of intellectual discourse. The follow-on naive-capitalist model was far too brutal and had consequences that they are still feeling–far fewer high-level publications in their own languages, far fewer high-quality scholarly publications in general (a significant problem in a small language group), and cultural costs that are hard to quantify but easy to identify as causing a sort of poverty in the intellectual culture.

What this has to do with the current revolutions may not be fully clear, since it’s about content scarcity, not content abundance. But the story is also about how a society can change its habits and patterns, and how quickly that can transform the culture. In those three or four years, among the unintended consequences was that quality content, and a society of ideas, was trumped by convenience, capital, and entertainment.