A few things should probably be noted in conjunction with GPeerReview. Firstly, it is not associated with Google in any immediate way, but merely lives on Google Code. Some blog posts suggest that it is an ‘official’ Google project, which as far as I can see it isn’t – at least not yet.

Secondly, the process envisioned by Mike Gashler, the main GPeerReview developer, departs from the traditional review process (as for example most journals use it) in several important (and positive) ways.

Here is the publishing process according to Gashler:

So, you’ve got a good idea. You’ve done some experiments, gathered some results, and written a paper. Now what?

1. The first thing you should do is pre-publish your work. Put your paper, your datasets, scripts, results, etc. on a public server. This does two things: 1- It ensures that science can move rapidly (without waiting for a response from a slow journal), and 2- It protects you from dishonest reviewers who might steal your ideas. [..] If some journal doesn’t permit works that were pre-published, you should not support that journal with your ideas anyway. Such journals will try to lock up your ideas rather than promote them. This is not good for you. Pre-publishing is good for you.)
2. Try to publish in a top-tier journal. A few publications with really good journals will benefit your resume/career much more than a lot of publications with so-so journals. It is well worth the extra effort required to get the endorsement of a respected journal. (Notice that up to this point nothing is different. Now, here comes the new stuff…)
3. Also submit your paper to several big-name endorsement organizations (EO’s). An EO is similar to a journal, but it doesn’t care whether or not a paper has already been published, it only cares how good the paper is. An EO doesn’t publish your paper, it just reviews and (hopefully) endorses your pre-publication copy. The “editor” of the EO will solicit the help of qualified reviewers to review your paper (just like a journal). He/she will coordinate double-blind reviews to ensure fairness. If the EO decides to reject your paper, they send you a private email with suggestions for improvement. If they accept it, they will send you a digitally-signed endorsement. (See more about EOs below.) Your c.v. (resume) should list all the endorsements that you obtain for each of your works. The idea that only the publisher can endorse a work is becoming antiquated. It is perfectly reasonable for many organizations to endorse a paper.

The concept of the endorsement organization is obviously the main innovation here (and I’m not saying that other aspects of GPeerReview are not innovative).

Why is it such a persuasive idea? Because it detaches evaluation from publication, two processes which have only been conflated in yesterday’s/today’s journal publishing system for historical and technological reasons. The infrastructure that publishers provide for disseminating your work is no longer needed – it’s a relic of the paper age. What’s still needed is peer review and peer endorsement in some form, but there is no practical reason why two separate processes should remain conflated into one once the technical requirements change. Authors can (and should) take care of making their ideas accessible and their institutions should support them with doing so. But evaluation is what that the community does – not the author herself (obviously), not her institution and not publishers.

From the website:

GPeerReview attempts to makes it easy for authors to seek post-publication endorsements of their works. We provide the following tools:

• A command-line tool to digitally sign endorsements (done and available).
• A web-based version of the signing tool (about 70% done).
• Client tools for analyzing endorsement graphs to establish credibility (in planning stages).
• Additional tools to facilitate the running of endorsement organizations (in the brain-storming stages).
• Tools for analyzing citation graphs (in the brain-storming stages).

On the onset, GPeerReview has two central components: a) a facilty to endorse a publication and digitally sign your endorsement and b) a tool to evaluate the endorsements you receive.

The second component is not entirely dissimilar to PageRank in that it makes ranking via dynamic social networks possible, and that the weight of my opinion as a reviewer is dependent on how others rate me. I won’t pretend to fully understand the mathematics behind it, but it seems plausible that the combination of by-name endorsements and numerical data provided by the peer reviewing network will provide a valuable indicator of quality – more valuable than what we have at the moment, at least.

From the Q & A:

If an endorsement comes from another scholar, then the scholar’s name determines the significance of the endorsement. Of course, there are too many scholars out there for anyone to recognize them all by name, but there are graph analysis techniques that can arguably provide valuable information. When researchers review and sign each others’ works, a decentralized social network is naturally formed. This network will eventually mirror the structure of the research community. If, for example, you wanted to determine how influential a particular scholar is with his research community, you could use an analysis technique that gives the information you want. We think the following algorithm might be a good choice:

1. Use a max-flow/min-cut algorithm.
2. Represent the individual being analyzed as the “source” node.
3. Select a number of well-known scholars of high reputation in your field. Represent each of these as a “sink” node.
4. Perform graph-cut to identify the sum strength of endorsements that would need to be hypothetically severed in order to separate the source from the sink.
5. Compare this value with those of other reputable researchers in your field.

As Gashler points out, another problem of the conflation of publishing with reviewing is that only one set of reviewers evaluates a publication, whethas a large and open community of people can review and endorse a publication with GPeerReview or a system built on similar principles.

So, how could this work from a practical point of view?

I think there’s another name for ‘Endorsement Organization’ and it’s Scholarly Society.

What makes it problematic for societies to publish journals is that they lack the infrastructure to act as a publisher and that maintaining such an infrastrucure is inefficient and costly. But with Gashler’s model they don’t need to actually store or archive anything. They let researchers typeset, proofread and upload their own material (or forego all of these things and risk being penalized) and instead act purely as EOs that leverage their social network qualities to provide something they are excellent at providing: a seal of quality.

At the same time this leaves room for disciplinary and institutional repositories in the system. The question of where something is stored is rather boring from a researcher’s perspective anyway – put it into your institutional repository if you want to make your librarian happy, just upload it to your website if you want to annoy him (with extra bonus points if it’s your private website), or go for something disciplinary if that feels like the best place.

What we really need are new ways of discussing our research with each other more rapidly and openly.

I think GPeerReview may be a big step into that direction.

Edit: Coverage of GPeerReview by Dean Giustini; more about open access and open peer review in Peter Suber’s blog.

Here’s a relatively unsorted list of thoughts that I’ve mentally penned since the last attendee headed for the exit of the Roy Lichtenstein Atrium.

We (as ‘the Open Access movement’, however vaguely that term is defined) need to…

• …do a much better job at bringing the scholarly societies to the table. The session on that topic was a good start, but it revealed that there is a communicative and perceptual rift between societies, who are by necessity conservative (they need to survive!), and OA activists, who can afford to be progressive. How will going OA impact a society’s membership? The common assumption is that members will defect if they no longer pay for the society journal with their fees, but there is no entirely conclusive evidence for that. Does going OA mean cutting off a source of revenue? Not always, as some societies heavily subsidize their print operation, meaning that going digital and OA may actually be a way of reducing costs for them.
• …look at more constructive ways of working with commercial publishers. In some disciplines and areas their role is more important than in others (in the natural sciences they are powerful because they frequently control impact, in the human sciences their expertise is important for the publishing process itself) and I don’t think it is impossible to publish or assure quality without them. But there is no reason whatsoever to shut them out and ignore their experience and ability to innovate – if they can find ways of recouping costs from non-subscription sources (author fees, advertising, associated services, POD…) we’ll all be better off if we have them with us.
• …convince funding agencies and politicians that publishing is not free, and that moving to a model in which access is both free and open means that costs must be recouped elsewhere. I’m entirely aware that this is widely known and accepted, but understandably public institutions are not jumping at the opportunity to spend additional money, especially while both the closed access system of subscriptions and the Open Access system of publishing subsidies must be funded in parallel.
  One strong argument for OA from a political perspective is global competition when it comes to cutting edge research. This presentation given at Berlin 6 by Solange Santos from Brazil on the immense scope of the SciELO network drove home that point: if we (in the [more] developed countries) do not move towards OA swiftly enough, and OA turns out be a significant catalyst for research innovation (as some claim it will be), we may find ourselves losing footing to developing countries that embrace OA more quickly at some point.
• …get librarians to work much more closely with researchers to support the publishing process. There has already been significant change in how libraries and librarians regard their role, but I think there is still a vast potential for change, especially here in Germany (my excuse for making blatant generalizations!).
  Librarians tend to see themselves as guardians of physical objects (books) and of digital objects which can be treated like physical objects (papers, dissertations, stuff that can be put into an institutional repository). In my (admittedly limited) view, they tend to be too far away from the researchers who desperately need someone to help them with the financial, legal and technical issues of publishing. I greatly recommend watching this presentation by David Weinberger for insight on the physical/virtual ‘objects of knowledge’ issue and some of the questions it raises.
• …and finally, approach researchers in a new, completely different way. There is still the belief among many involved in financing, supporting and disseminating research that those who undertake it have both the ability and the motivation to “move to open access” by themselves. I don’t believe that this is true. Many junior researchers who might be in favor of OA cannot choose freely where they want to publish, because the wrong choice is a risk to their career. Many senior researchers who have the influence and standing in their discipline to drive a paradigm shift do not embrace OA because the formats, publishing channels and procedures involved are unfamiliar and appear unreliable to them. But at the core, neither of these issues is decisive.
  The pivotal problem is that most researchers, regardless of where they stand on the career ladder, are not impacted personally by whether or not something is Open Access, and that their perspective as individuals, and not the common good, shapes their views. As Steve Anderson (I believe) pointed out in his talk at the conference, there’s a difference between free beer and free speech, between “I don’t have to pay for it” and “It’s in the public domain”. Researchers are by and large shielded from the subscription costs of the closed access system by their institutions and I honestly believe that many of them are not too concerned over the privatization of public assets that follows from the entrenched system. The business of business is business, to quote Milton Friedman, and similarly in our modern, globalized research environment the business of research is research, and not a whole lot of anything else.
  Let me drive the economic analogy a bit further. Those in favor of deregulating the world’s financial systems often cited the self-interest of lending institutions to protect shareholder’s equity (Alan Greespan) as a strong fail-safe, one that made third-party intervention unnecessary. But just as investors (primarily) focus on short-term profit, researchers (primarily) focus on immediate career benefits, not on the public good or the taxpayer’s burden.
  Open Access is a societal issue and not an individual one, therefore we cannot expect individuals with individual interests to be the driving force behind it, but only institutions with collective interests. Only by providing the right mix of incentives, mandates and support (both financially and in terms of know-how) can we get the horse to drink, to use Dieter Stein’s metaphor of the horse and the trough.

So how do the parties listed above succeed at supporting Open Access?

I believe that one very effective way of enabling OA in the long term is to push for entirely new forms of publishing, forms that are ‘OA by nature’, such as blogs and wikis. The entrenched forms are conceptually associated with the entrenched system and it will probably be harder to disassociate the one from the other than to popularize entirely new forms of science communication (i.e. ‘journal’ and ‘article’ are conceptually associated with ‘paper’, ‘commercial publisher’ and ‘subscription’, while ‘blog’ and ‘wiki’ aren’t).

New forms of scholarly communication that have novel advantages over existing forms will be adopted not because they are open (because, as outlined above, by itself that hardly matters) but because they offer specific benefits to the individual scholar. Obviously they will exist side by side with established forms. But they could act as a catalyst that raises awareness among researchers for the benefits of Open Access, because the reach and openness of hypertext publishing is what makes it so attractive.

I think it’s necessary to work on many different fronts at once. One thing that institutions (specifically those with a disciplinary anchoring) should do, is to give all their researchers easy-to-use tools for personal hypertext publishing.

What about starting a wiki-based platform similar to OpenWetWare as an inter-institute channel to facilitate collaboration among Max Planck institutes? Why does no major university (that I know of, at least) operate an institutional blog planet so that I can see what my colleagues in psychology, economics or computer sciences are working on? If these tools can facilitate collaboration at transnational companies, why not at universities and research institutes, which should be about collaboration by default?

Update: I have found a blog planet at an academic institution after all – at the University of Cape Town. I think this underscores my point about so-called ‘developing countries’ being more innovate than we are. While I’m sure there are other examples of institutional blog aggregators in different countries, I am not aware of a single one at a German university.

I’ve been studying blogging for quite a while now, and specifically the use of blogs in academia and research is something I want to investigate in more detail in the near future. Science blogging will be a central topic in the session I’ll moderate at the Berlin 6 Conference in two weeks and I’ve recently written a grant proposal for a project that would focus specifically on blogging in the humanities and social sciences.

Among other things, I would like to investigate

• hard factors (based on natural data, i.e. the blogs themselves) such as
  • who blogs and who doesn’t (age, disciplinary background etc),
  • what, when and how academics blog (formal properties such as post length, type of content words used, tags, stylistic features, use of links and quotes, visual presentation, frequency of posts etc)
• soft factors (based on interviews with practitioners and meta-discussions on blogging) such as
  • how academics characterize their activity relative to ‘general-purpose’ bloggers (using similar questions, this could be compared to the results of the Pew and Technorati studies)
  • how blogs are conceptualized by their users (as scientific publications, scribbled public notes, discussions with peers, personal musings, a mixture of all these things)
  • what the perceived advantages and disadvantages of blogs are (risk of being snubbed, losing credibility etc)

I’ve used a similar bivariate approach in my PhD thesis to look at hard and soft factors of how corporate blogs are written. Language data found on the Internet is difficult to evaluate without contextual information and my impression is that the ethnographic approach (interviewing practitioners, meta-discussion) often holds the key for truly understanding what is going on. Both talking to bloggers and applying standard language analysis techniques to the data they produce is likely to yield the most comprehensive picture of academic blogging and where it is headed in the future.

Blogs are used for many different reasons by many different people and therefore a lot of misunderstandings arise about their usage. A senior scholar may think that since adolescents use blogs to document their lives and to foster social relationships, blogs are inappropriate for academic purposes – such personal concerns have nothing to do with serious scholarship after all (right?).

The artificiality with which researchers (and ‘professionals’ in other contexts) separate their personal and professional lives is an interesting discussion point in itself, but even assuming that the two remain detached from one another, a blog entry can be as ‘serious’ as any journal article.

At the core, a blog is merely a personal publishing platform for distributing content in a sequential mode. The latter – sequentiality – is what truly distinguishes the blog from other publishing technologies (wikis, which connect information syntagmatically via hyperlinks; general websites, which often have a hierarchical content structure). Blogs, along with wikis and other hypertext publishing platforms, have the potential to become extremely powerful tools for research because they are so versatile and simple to use. Simplicity and ease of use are often-underestimated but crucial factors when it comes to the apoption of new technologies.