There was a problem with a spam-blocking plugin in the back-end of this blog that was preventing people from making comments. The problem is now fixed, so if you feel like commenting on any recent posts, have at it.
There was a problem with a spam-blocking plugin in the back-end of this blog that was preventing people from making comments. The problem is now fixed, so if you feel like commenting on any recent posts, have at it.
Libraryland is a-buzz about a new role we can play in the pursuit of scientific knowledge: data curation. Data curation serves, in particular, the new scientific methodology that goes under the name e-science. E-science involves the collection of data sets which are made widely available to the research community. Researchers then “mine” these data sets by using automated systems to find statistically significant relationships within the data. The library’s role is to curate the data, i.e., identify, acquire, and manage the data sets through the course of their life cycle. As exciting as this new methodology is, one should be aware of its weaknesses. E-science can be a valuable addition to traditional scientific methodology, but by itself, it is no panacea.
In a commentary entitled “Implications of the Principle of Question Propagation for Comparative-Effectiveness and ‘Data Mining’ Research” in the Journal of the American Medical Association, 35(3), 2011, Mia and Benjamin Djulbegovic argue that data mining does not provide definitive answers to research questions. Instead, it should be considered merely a hypothesis-generating technique. Their first point already had been demonstrated vividly by a piece of data mining research entitled “Testing Multiple Statistical Hypotheses Resulted in Spurious Associations: A Study of Astrological Signs and Health” published in the Journal of Clinical Epidemiology, 59(9), 2006 by Peter Austin et al. Austin et al.’s research showed that residents of Ontario, Canada who were born under the astrological sign of Leo had a higher chance of suffering from a gastrointestinal hemorrhage than others in the population, and those born under the sign Sagittarius had a higher probability of being hospitalized for a humerus fracture. These results were statistically significant, even after being tested against an independent validation cohort. The study “emphasizes the hazards of testing multiple, non-prespecified hypotheses.” In other words, it warns us that given an enough data points, one can, after the fact, find any number of ways to connect them.
The second point in Djulbegovic and Djulbegovic, that data mining should be used as a hypothesis-generating technique, is, on the other hand, undermined by Austin et al. Austin et al. point out that the statistical methods that are at the heart of data mining are not able to distinguish real from spurious associations. Data mining employs the automated examination of enormous bodies of data. Its usefulness is thought to be proportional to the size of the data set that it collates; however, as the data set becomes larger and as the number of attributes that serve as potential relata increases, the number of potential relationships increases exponentially. Importantly, the number of spurious associations also increases. With enough data, no significance test will be stringent enough to provide assurance against the kind of results found in Austin et al. What is needed, according to Austin et al. is a “pre-specified plausible hypothesis.” For statistical analysis to be useful, the researcher must begin with a hypothesis, preferably a plausible one, if the research is to be valuable.
What exactly is a pre-specified plausible hypothesis and how can we generate it if data mining can’t do that for us? The question was posed some sixty years ago by the philosopher Nelson Goodman using different terms: Goodman believed that a critical question for epistemology was to distinguish between “projectible and non-projectible hypotheses.” One can more or less replace “pre-specified plausible hypothesis” with Goodman’s term “projectible hypothesis.” According to Goodman, when we seek to understand what hypothesis is (or is not) projectible, we do not come to the problem “empty-headed but with some stock of knowledge” which we use to determine what is (or is not) projectible. Projectible hypotheses will be those which do not conflict with other hypotheses that have been supported in the past. They will commonly use the same terminology of previously supported hypotheses. The terminology appearing in the hypotheses will have become “entrenched” in the language. This goes a long distance toward explaining why we don’t find the link between one’s astrological sign and medical conditions plausible. Twenty-first century Western medicine is not accustomed to linking astrological signs to ailments and so must find any hypothesis that does so implausible.
If Goodman is correct, then data mining is of little use without an historical understanding of the field of science to which the data pertains. Library administrators should keep this in mind when allocating resources. Clearly, purchasing data sets is a necessary part of serving our research patrons, but the emphasis must be not on the mere accumulation of data, it must be on the selection of data that is critical to continuing the scientific discourse. While data sets that distinguish astrological signs are clearly insignificant for medicine, there are many other attributes that form the basis of data sets that are more or less reasonable. Librarians must be able to perform the complex task of distinguishing the more from the less. It is the curation of data that is important, i.e., the acquisition and management of data sets through the whole of its life cycle; and most importantly, the curation of data sets that are of interest and value to the scholarly and research community.
Here, we have another argument for allocating library resources to pay for librarians with deep subject expertise. As e-science develops, vendors will make more and more data sets available, regardless of their actual worth to researchers. To effectively choose the data sets that are of value, librarians must have a thorough understanding of the research needs of their patrons. To do this, they must have a deep understanding of the field. Unfortunately, with the excitement swirling around e-science, the mere access to large data sets threatens to become the be-all and end-all in collection management. If we aren’t careful, we may find ourselves with mountains of data from which everything and nothing can be concluded.
I’m talking with Jennifer Szunko, Clarion Services Director. Jennifer offered to be interviewed for Library Juice to talk about Clarion’s services in doing paid reviews for authors and publishers.
Thanks for being willing to do this interview. Jennifer, can you explain a bit about what you do at Clarion, and how it is different from Foreword Magazine, which I understand is the parent company?
Thank you for inviting me.
For the past 14 years ForeWord Reviews has offered pre-publication reviews of good books from independent publishers in our print magazine. Like most magazines, it is supported by advertising. Ten years ago, librarians were asking for more reviews than what we could accommodate in the magazine and with the explosion of books being published we were receiving hundreds of books each month that we just could not fit into the magazine–the Clarion Review fee-based service was born. I manage the orders that come in for books to be reviewed for a fee and work with the same pool of reviewers as we use in the magazine.
The reviews in our magazine are all pre-publication and are all for good books–those reviews are all very positive. But there are millions of authors that would still want a review but their books just didn’t qualify–either due to the publication date or the editorial content didn’t align with the magazine’s calendar, or the book just hasn’t captured the attention of an editor. That is when the author will order a Clarion Review.
Thanks for that description. Just to clarify, when Foreword Magazine was new and before Clarion Review was started, did Foreword Magazine offer paid reviews?
No, we have never charged for reviews that appear in the magazine. We only started the fee-based review service to accommodate the “overflow” of requests from authors to get their book reviewed. And it offers librarians many more reviews–all of which are archived with Bowker’s Books-In-Print online, Baker & Taylor’s Titlesource 3, and Ingram’s iPage, in addition to Google Books and the Foreword Reviews Website.
Interesting. Where else might librarians or readers encounter the reviews?
The authors will also use the review (if its a positive review) in the marketing of their book, on the back cover, or on their own Website. All of our reviews reflect an honest and unbiased look at a book, however, we only publish reviews of the best books in the magazine. You will find everything from a one star to a five star review on our website. The difference between the reviews in the magazine and a Clarion Review is that the Clarion Review gives a critical analysis of the book–useful information for librarians and booksellers.
So it is possible for a paid review to receive a one-star rating? I can imagine that that would result in some unhappy customers.
I recently pulled a report of all of our reviews and found an almost perfect bell curve. We have mostly three-star reviews but we about the same amount of one-star reviews as five-star reviews. And the two and four star reviews are balanced, too. Its never easy delivering a negative report to an author but more often than not they appreciate the honesty. With digital technology, it’s easy to implement many of the changes that are mentioned in the review; so the author ends up producing a better book. So everyone wins!
Thanks for the info on stats. That is reassuring. About how many paid reviews do you publish a month? And just to clarify, you’re talking about paid reviews there, right?
Yes, the Clarion Reviews are purchased by the author and those are the only reviews that receive the star rating. We publish about 70-90 reviews on our website each month and share those with the licensees I mentioned earlier. We do give the author the opportunity to “kill” their review if they feel it is too negative. Most recognize the value of a professional review so they don’t go that route.
Even though the highest proportion get only three out of five stars. That is a more credible system than I thought, I have to admit.
I think it’s to Foreword’s credit that Clarion Review is open about selling paid reviews. However, I wonder if there is any disclaimer to that effect in the reviews that are published through these licensees, and whether at least Clarion is identified as the source.
Clarion is identified as the source on all the reviews that we write for a fee. Our integrity is extremely important and if anyone thought the reviews were somehow compromised because of the fee, we wouldn’t have survived this long. All of our reviews are objective opinions about a book, and there has been some reticence by a few publishers or authors who prefer to do business using an outdated business model, one that can no longer support itself due to declining advertising dollars. I don’t know any librarians who criticize the paid review service. Librarians are the reason we added the additional review service; they want as many quality reviews as they can get and look to us as a trusted resource. Our brand speaks quality and they can depend on us, and they are the biggest users of subscription services at our licensing partners…who are also dying for all the reviews we can provide them–regardless of whether the funding comes from ad dollars or a straight up fee.
Are you familiar with the review service BookNews?
BookNews also produces brief reviews that are published by book jobbers for the benefit of collection development librarians, though I suppose they may focus on the academic market. Their reviews are all solicited; as a publisher I receive requests for titles from them for review. I believe their source of funding is the book vendors themselves. I don’t know about the quantity of reviews that they publish, but I do know that they select the books that they choose to review.
Interestingly, ForeWord was the first to offer a fee-based review service and took a great deal of grief for it for several years but now we are one of many. To get around the stigma of a “fee-based review” other companies call it a publicity fee or a marketing fee but when you get to the bottom of it –we are all offering a professional service and professionals are paid to deliver a professional product. The BookNews model sounds more like the model for our magazine–our editor solicits for books based on our editorial calendar.
Regarding the question of the disclaimer… While Clarion Review is out there advertising paid reviews, do you think that most librarians or readers will be clued into that fact when a review is credited as coming from Clarion? I think this is an important point, because even if the highest proportion of reviews are mediocre, there has to be a market effect on the reviews Clarion gives in order for the service to be financially successful. Yes, authors want credible reviews, but they also want reviews that will lead to sales, which is the purpose of it from their point of view. Normally, or at least in the “outdated business model,” the appearance of a review usually constitutes a recommendation in and of itself. If it weren’t for the fact that so many of the paid reviews come from self-published authors, I think the bell curve you cite would be less troubling. I do find it a little hard to believe that given the nature of the customer base that so many books would be judged as being basically pretty good.
That’s an interesting position. Its not an argument that I have come up against. If you look up the word review, I think you will find it to say something like a critical analysis and not a recommendation. So maybe that is where the distinction lies. Clarion reviews reflect strict editorial standards and positive consideration is given to well-written and produced books–anything less than that must be mentioned in our reviews. It probably helps that there is at least a one person buffer between the reviewer and the author–the reviewer has no emotional stake in the transaction. We don’t hesitate to have a reviewer rewrite a review that sounds too much like an attempt to sell the book–we exist to provide honest information on books. That doesn’t always make us popular with authors or publishers.
What is the profile of the authors and publishers who use the paid review service? Do well-known publishers use them? What is the proportion of self-published authors using the service, as far as you know?
And regarding the judgment of quality, it is a relative thing. It just seems to me that in a paid review situation, if there are a lot of self-published authors, in order to maintain a bell curve the standards are going to have to be lower than they would be in a selective review source.
Good point about the self-published author as a big part of our client base. We have seen a tremendous improvement in the quality of the books from self-published authors in the past two years (and we have been doing this for ten). Remember that we do offer the option to “kill” the review and it is more likely that a one-star review will be put to death than a five-star review; the numbers do skew a bit in that sense.
How many reviews get killed relative to the 70 to 90 published a month?
ForeWord has always only reviewed books from independent publishers. We have not reviewed books from any of the Big Six or their imprints.
What about the proportion of reviews going to self-published authors versus books published by small publishing houses? (Small publishing houses being selective by necessity.)
I have not tracked the number of reviews that are “killed” each month but I would guess more than half of the one and two star reviews do not survive. So, yes we do actually write many more negative reviews than five-star.
Of the one-star reviews that do not survive, are you counting them in the bell-curve cited earlier or not?
Reaching the small press market with the Clarion product may well be our biggest challenge but we have kept very busy with self-published authors which make up about three-quarters of our fee-for-review service. The landscape is changing quickly and with the exponential growth in book production–authors and publishers are looking for ways to get noticed. That is true for self-published authors and those that have a contract with a small publishing house.
The bell curve that I mentioned earlier is only the reviews that are actually posted on our website.
Regarding the stats on the one-star reviews, that is encouraging from my point of view.
I have something to disclose that leads me to a question. About ten years ago I was very active in the group Alternatives in Print, which was a part of the Social Responsibilities Round Table of ALA. A major focus for us was advocating alternative press publishers, which for us meant small publishers publishing books that we felt were kept out of the mainstream market because of their political viewpoints. A big part of what we wanted was for books from these publishers to be more widely reviewed, and we were looking for ways to accomplish that. We were often asked for advice by these small publishing houses about how they could get their books into libraries, and reviews were a part of the answer. I think it was in that context that I first had contact with someone at Foreword Magazine, so I believe that I am probably one of the librarians you’re counting as asking for more reviews from these hidden sources. From my point of view, however, what Clarion is doing is not exactly what I was asking for (just to speak for myself), in that what I was talking about then had to do with the commercial nature of publishing, and the role of money in the marketing of books to libraries. What we were advocating in AIP was for a more non-commercial review sector that would compensate for the advantage that major publishers have in getting reviewed, by virtue of their economic power. The publishers we tended to advocate were publishing a lot of anti-capitalist books, and to them I think something like a paid review service would be anathema. Our analysis about the distortion in the market had to do with the political nature of the books we wanted to advocate, whereas for you I would imagine it has to do with the place of the little guy in the overall scheme. How would you approach small press publishers who may have a bias against something like paid reviews for these kind of idealistic reasons? I mean, to someone with that kind of a point of view, it smacks of the Better Business Bureau pay to play scandal.
I do think we are on the same page in the end–librarians want to be notified of the true contents in books and authors want visibility and an honest evaluation. This is true for all publishers, but you’re right: the bigger budgets get the bigger splash. Foreword is about the little guy and bringing attention to good books. My question would be…how would a small business owner support their staff if all of their product was offered at no charge? In the ideal world we could charge an extremely high rate to our 15,000 librarian subscribers for the privilege of reading our reviews–in reality, we would not survive. Instead we choose a per piece model so we can offer the quality we demand at a fair price to our customer. Every Clarion Review is worked on by four editors before it is returned to the publisher or author–that type of commitment to quality comes at a price. We never set out to be all things to all people but we do have a growing body of publishers and authors that value our service. There are non-commercial review sources but they are inconsistent and their quality is questionable–they don’t have anything to lose. We guarantee our quality and stand behind our reviewers opinions.
I disagree that non-commercial sources of reviews tend to be unreliable. Being in academia, where all the reviews are produced non-commercially, from my perspective the situation is the reverse of what you say. As a librarian who has worked in all types of libraries, I would just offer that in my opinion there is no demand for reviews of a great number of self-published books. I see nothing wrong with a business model that requires reviewers to be more selective; including more small press titles can easily fall within that framework if there were stronger regulations against the kind of pay to play practices that result in reviews not being given in sources like PW or LJ where the publisher doesn’t purchase ad space. I have never seen a need for the reviews that result from a pay to play review service. But I can only speak for myself – obviously you have access to more objective information about the market for reviews.
So, I have offered my opinion now. Before we say goodbye is there anything else you’d like to add?
Just that I have enjoyed our discussion and its a very exciting time in the publishing industry and we are thrilled to be a part of it all.
I want to thank you for all the information you provided about Clarion Review. I learned a lot from our conversation and enjoyed it as well.
CALL FOR PAPERS
2nd Milwaukee Conference on the Ethics of Information Organization
June 15 – 16, 2012
Information organization, like other major functions of the information professions, faces many ethical challenges. In our literature, ethical concerns have been raised with regard to, topics such as, the role of national and international tools and standards, provision of subject access to information, deprofessionalization and outsourcing, education of professionals, and the effects of globalization. These issues and many others like them have serious implications for quality and equity in information access. The Information Organization Research Group and the Center for Information Policy Research of the School of Information Studies at the University of Wisconsin-Milwaukee join in presenting this second conference to address the ethics of information organization.
Like the first Ethics of Information Organization conference held in Milwaukee May 2009, this conference (June 2012) welcomes papers on ethics and any element of information organization from cataloging standards to tagging; subject access; technology; the profession; cultural, economic, political, corporate, international, multicultural and multilingual aspects.
INVITED SPEAKERS WILL INCLUDE:
Opening Speaker: Jens-Erik Mai
University of Toronto
Closing Speaker: Richard Smiraglia
University of Wisconsin-Milwaukee
Other invited speakers will be announced
We invite submission of proposals for papers which will include: name(s) of presenter(s), title(s), affiliation(s), contact information and abstracts of 300-500 words. Presentations will be 20 minutes. Time will be set aside for questions as well as broader discussion. All abstracts will be published on the Web site of the UW-Milwaukee Information Organization Research Group. Full papers will be published in a special issue of Knowledge Organization.
ABSTRACTS DUE: February 15, 2012
NOTIFICATION OF ACCEPTANCE BY: March 15, 2012
FULL PAPERS DUE: July 15, 2012
Submit proposals via email to: Hope A Olson, Conference Chair (holson [at] uwm.edu)
CFP poster available here
Last month in Varanasi, India, the First International Conference of Lokavidya Jan Andolan, or the Peoples’ Knowledge Movement, was held. The original call for participation talked about the situation of the displacement, environmental destruction, and poverty experienced by people throughout India:
All these people, the displaced, the communities they belong to, have never gone to college and live by the knowledge they posses[s], called lokavidya, which they have acquired from elders, from peers, in the community, at the site of work, through experiments and by their own genius. […] In fact lokavidya, that is people’s knowledge, skills, ways of thinking, values, methods of organization, aesthetic and ethical sensibilities, in short, their world of knowledge as a part of their own world, is the main source of their strength. […] It is important to understand that the emancipatory pathways today traverse through the world of knowledge. The Lokavidya standpoint is the people’s standpoint in the Age of Information.
Looking around online a little, I came across a post from a 2008 gathering, which noted, among other things:
“Most people, peasants, artisans, adivasis [indigenous people], very small shop-keepers and women have never been to a college or a university, but they have their own extensive knowledge.”
“The reason for the very bad condition of their life is that their knowledge, lokavidya, is not organized. Their knowledge gets no recognition in politics, in the big bazar, in leading cultural institutions and in the universities. Actually the power centers of the society refuse to give the status of knowledge to lokavidya.”
“So long as lokavidya is not organized, the lokavidyadhars will not be able to effectively intervene in the public realm. Lokavidyadhar samaj needs to take initiative to organise lokavidya under its own leadership, then only can they command respect and get rid of poverty.”
I don’t know enough about the Indian context, but from the U.S. I can’t see that poverty and other manifestations of inequality could be traced to a lack of “organization” of knowledge. Here, voices on the left celebrate the knowledge, traditions, values, and culture of unprivileged communities (of which they may or may not be part). But I don’t think anyone is suggesting that society would change appreciably if only their “lokavidya” were respected by the centers of power.
This also makes me think of a workshop/discussion I attended at the Critical Resistance conference in 2008, where people were riffing on a variety of facts and ideas about prison and justice. Finally the longtime activist Kai Barrow took the floor and said something like, “Okay, now, what are we going to do when we leave here? We have all the information. We don’t need any more information.”
In other words, information (which leads to knowledge – obviously these are two different things) is not emancipatory in and of itself. But I think Kai’s plea does speak to an aspect of this lokavidya concept, a sort of active quality, coming as it does “from elders, from peers, in the community, at the site of work, through experiments.” It’s not just book-reading that’ll learn you. And given that Kai wasn’t actually suggesting that people take up pickaxes and start hacking away at the nearest prison walls, even in an active activism, the work of a prison abolitionist would involve creating new knowledge as a means to making real change.
Coincidentally, I’ve been reading Siddhartha Deb’s The Beautiful and the Damned: A Portrait of the New India, and in a chapter about Bangalore and engineers and IT, Deb writes:
There is also something Brahminical in the very way engineers perceive their work around computers, if by Brahminical one means the idea of exclusive access to knowledge that cannot be shared with commoners. There is no glamour in India, for instance, associated with being a civil engineer, and in this it differs remarkably from countries in the West, where, through the nineteenth and a great part of the twentieth century, the civil engineer was celebrated for his rugged masculinity, especially in the way he dominated nature by building dams and bridges.
Today’s Indian middle class, in contrast, celebrates the engineer-entrepreneur who makes money or the engineer-functionary who sits at a workstation. The cubicle is clean, air-conditioned and unpolluted, while the factory is dirty and physical. The cubicle is Brahminical, the factory is Sudra, the realm of the low-caste craftsperson. (p. 99-100)
Anyway, to conclude this sort of stream-of-conscious post – despite my relative unfamiliarity with India, I wanted to write up something here because I haven’t been able to get the idea of this lokavidya conference out of my head.
Julie Bosman writes in the New York Times about a logical new development in book publishing: finer quality, fancy print books to compete with e-books. What is not expected about this, to me, is that major publishers are moving in this direction, rather than boutique houses that specialize in it.