The Shatzkin Files

Merchandising ebooks is a problem not really solved yet

Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Share on RedditShare on StumbleUponEmail this to someone

I have always been in the process of reading at least one book since I was about 8 years old. When I was a little kid, I’d find them in the house (Dad was in publishing) or at the library in my home village of Croton-on-Hudson or in the school library. Sometimes extraordinary measures delivered a lot of reading material. On the fourth to the last day of second grade, I got the chicken pox and was in bed for a couple of weeks. I had already developed an affinity for a Random House series of children’s books on American history called Landmark Books, which are still available. Dad knew the person at the printer responsible for the Random House account and a box of 40 of them arrived the day after I was diagnosed and was completely read through by the time I was back on my feet.

When I was in junior high school, I found that a big drug store in the retail space at 42nd and Vanderbilt in Grand Central Station had a massive selection of mass-market paperbacks and that became a shopping destination for me for a while.

As an adult, the shopping and discovery moved to bookstores. And although I did occasionally get my ideas of what to read next from book reviews or friends’ recommendations, usually I just shopped. I would go browse American history or biography or sports (baseball always had its own shelves within sports).

It never took me much time to find what I wanted to read next until I started reading ebooks.

In the pre-Kindle ebook era, I was a captive of the Palm Digital store, because I read on a Palm and their commercial approach was to not allow other retailers to sell their format. The choices were limited because the publishers before the arrival of Kindle were reluctant to make the investments required to deliver ebooks to me and the four other people who read them at the time. That changed immediately when Kindle arrived and, because of Kindle and the other major formats that have hit the marketplace since then, the choices are robust. Just about every new book I’d want to read is available for my device of choice (the iPhone) and the digitization of the backlist just carries on going deeper and deeper into publishers’ repositories.

But the merchandising, at least for somebody who shops on the iPhone (it’s a bit better through the ereading devices or PCs), leaves a lot to be desired. My shopping experiences are actually a bit of a random walk. I ask my ebook retailer to show me books by category and, since my categories don’t change much (and haven’t since I was a kid) I tend to see the same books over and over again, far too many of which I have already read (perhaps in somebody else’s format.)

A short time ago I was shopping for my next read on the iPhone. I started out shopping with Kindle and then Nook and a few minutes on each of their mobile sites didn’t turn up anything that moved me. Then at Google Ebooks I found “Making of the President 1968” by Theodore White. That was definitely one I wanted to read. I bought it and I’m in the middle of it.

There is no particular guarantee that I’ll find my next book on Google. I haven’t found any clear pattern yet among the four stores I shop regularly (Kobo being the fourth). Obviously, if I know I want to read another James Patterson or John Locke thriller, any of them would deliver it to me quickly and painlessly in response to a search. It is when I am hunting by subject that the search returns seem to be pot luck. I’m probably not making it any easier on the retailers by spreading my shopping around; if any of them actually did have a good engine to take my purchasing and reading profile and make the next great recommendation, I’d be screwing it up by spreading around my data.

All of this underscores how difficult is the challenge being faced by Bookish in the US and aNobii in the UK, two “find what to read next” sites financed by major publishers. And they join a long line of sites that have tried to build recommendations and community conversation around what people are reading: Goodreads, Shelfari, Library Thing, and the new ebook platform, Copia.

It happens that our office is now going through the exercise of placing the book of “The Shatzkin Files” on platforms other than its originator, Kobo. (Kobo’s 60-day exclusive is about up.) When we encountered a limit of seven keywords in loading process for Kindle, I inquired about it. Why limit this, I wondered?

I got a good answer when I asked. It turns out that any author or publisher’s inclination would be to put in lots and lots of keywords. That was my intention. I was going to take every keyword from every post and put it in for the book. But, on reflection, as my friend at Amazon pointed out, that really wouldn’t be helpful to the reader who was searching. The fact that one blog post is about a holocaust survivor doesn’t mean that somebody searching under that topic would want my book, of which more than 99% is about things totally unrelated.

It turns out that Amazon uses algorithms created by full text searching to enhance what they can deliver in response to searches in ways that the publisher and author would not necessarily think about when creating metadata. As an example, he pointed to a book that you’ll discover on Amazon if you search  for “erasure coding”, a term of art that might very well not have been included by any author or publisher inserting keywords but which their more sophisticated methods enable you to use for discovery.

My friend at Amazon didn’t say this, and maybe I’m reading too much into what they do, but it almost seems like the keywords we put in could be superfluous and the capabilities they have through full-text analysis and algorithms actually govern what is discovered. Of course, if the solicitation of keywords from authors and publishers is a placebo, that’s not something I’d expect them to reveal.

I was just looking for “American history” when I found “Making of the President 1968” on Google (and didn’t find it anyplace else in the time I allotted to look.) So Amazon’s sophisticated capabilities didn’t deliver it to me and now their engine doesn’t know that this was a book I wanted because I bought it someplace else.

But I’m really glad I found this book, which was probably pretty recently made available in ebook form. I was active in that campaign and at the Democratic Convention in Chicago, where I was Pierre Salinger’s assistant on the first McGovern campaign. (George McGovern declared late to give the Bobby Kennedy supporters who couldn’t abide Gene McCarthy a place to go. I had been one of those; I left the Ambassador Hotel an hour before Kennedy was shot on June 4, 1968 because the security was tight and I couldn’t get into the party. Ironic.) The author of the Making of the President books, Theodore White, was a friend of Salinger’s and I met him at the convention. But I’m saving the stories of that campaign for another post on another day.

Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Share on RedditShare on StumbleUponEmail this to someone

  Back to blog

  • iliad1954

    This is the ballgame. In the marketplace, an e-book is nothing but its metadata and metadata doesn't serve too well for merchandising. But I wonder if, in the short run at any rate, it might not often work like this: You're interested in the 1968 Democratic Convention?  Here's all the books about that: Word of mouth, real or virtual, will always be compelling.

    • Cool. Except Making of the President 1968 isn't on the list. I guess that's

      because it was about the whole year, not just about the convention in

      Chicago (which made it unimportant for this particular bit of curation.)


      • iliad1954

        Theodore White's book is on that page, just way down the list as “Background Reading.” And I'll edit it to link to the e-book edition of the book.

      • Thanks. Sorry I didn't go far enough to find out. I was confirming with

        alphabetical order.

        I don't disagree with you that community curation is ultimately the key. The

        problem, of course, is that I wasn't looking for a book on 1968. There's a

        good chance that I wanted American history, president-heavy, anytime between

        1789 and 1980.

        The work you're doing is essential. But it will need to be rolled up in

        different ways by other curators to be part of the ultimate solution

        to *my* problem.

        However, you're solving a lot of other problems and you're creating one of

        the essential building blocks.

        Things will be very different 10 or 20 years from now.


      • Erica Frank

        The digital slushpile is the next big issue for ebook sellers, be those authors, publishers, or websites, to tackle–and it's an area where mainstream publishers have an advantage; they have practice writing pitches and creating curated lists of similarly-themed books.

        Since I don't deal with DRM, I'm outside the standard marketing range for new books; I wind up crawling obscure blogs looking for recommendations and poking around at indie publisher websites. I've found a few that I like, but nothing that jumps out as My First Place To Look For New Reads. Not even if a few of them got together and combined their lists.

        I think some of the problem is sheer volume–when 15,000 books are published in a year, picking the top 1000 to list in magazines is (relatively) simple; picking the top 25 out of that 1000 as extra-special recommendations is easy. It's possible the numbers won't be entirely accurate; maybe the “top 25” will, by whatever criteria, include only the top 15, and 10 more from the top 50. But it'll be clear enough that these are “highly recommended,” and popular enough to confirm or refute that opinion with someone who knows your specific tastes.

        When a million ebooks are being published every year, ranging from 750-word short stories to 150,000-word epic novels, that kind of curation is impossible. No one person or team is even going to read the complete list of titles & authors, much less try to find the good stuff. Forget the needles & haystacks; we're now looking for the diamond dust sprinkled on the beach.

      • You're right that it will be a long time, if ever, before we get to

        mechanisms that can process everything out there. But I think crowd-sourced

        creation in niches — genres and subjects — will work when some tools are

        put in place. I look at that as one of the roads bound to be paved someday,

        even though it isn't yet.


  • Mike, what about going outside the ebook retailing channel? There is a lot going on at web sites like goodreads. I sometimes find new ideas for books to read by perusing their “most read this week” sections for different genres. History books are here for example

    Also, some local booksellers that have partnered with Google have better web sites for finding new ebooks than any of the majors — it's like walking into a typical small bookshop but online. For example, one store near me, Porter Square Books, offers their “staff picks” as ebooks online in addition to physical copies in-store. See http://www.portersquarebooks.c… for example.

    • Thanks for the referral. Having more places to look might be helpful. But I

      don't think these “solve” the problem any more than the ebookstores do.


  • I think it's in The Making of the President 1968 that White tells a story about meeting an old-timer who had attended many such conventions. The two leading candidates for nomination were Nixon and Rockefeller. 'Who's going to be nominated?' White asked the old guy. 'Nixon.' 'Why?' 'Because when you look at Rockefeller you can't see his eyes.'

    To my mind that says a lot about politics, all over the world, and maybe something about merchandising too.

    • I read the Republican convention part already and don't remember that

      anecdote. Maybe somewhere else and maybe I glided past it.

      At some point I'll write my personal recollections of the 1968 election.

      I'll preview it to tell you that Sonny Bono came to UCLA to speak on behalf

      of Rockefeller to a bunch of students before the Republican convention. That

      was back when he would have been thought of as a hippie freak, long before

      anybody would suspect that he'd be a conservative Republican Congressman!


  • Allan Connery

    You might try adding The Making of the President 1968 to your Amazon Wish List. In my experience, that will produce a surge of similar books in your Amazon recommendations.

    • That's useful advice. Sounds like the equivalent of creating a

      music channel on Pandora. Thanks.


  • Yes, it's a big challenge. When I used to go to a bookstore I would go to my favorite sections and there were small enough that I could browse the offerings in 5 or 10 minutes and grab a new author who looked interesting and the new book by an author I loved. Now there's just too much to choose from.

    That said, there's a lot of potential out there for data mining on ebooks and online book shopping in general. I hope for the day when I can have recommendations come to me as – Here's a new book by an author you bought from. Here's some interesting stuff in your favorite genres and here's some books you might like based on the reading you do.

    iTunes is probably the most useful for me, they have opened my taste to new music with the recommendations and they support 'here's something new' with 'you liked X so we think you might like Y”
    Amazon is getting there, but the other on line retailers don't seem to have jumped on the bandwagon yet.

    Great post.

  • Jenny Ruhl

    I find the best books on the New Books shelf at my library. And not just because I'm cheap, but because our local librarians, God bless them,  find treasures reviewed in Library Journal that I'd otherwise never hear about or find in the Top-40 chain bookstores or even the few local “We only stock it if it's reviewed in the NYTimes” indies.

    Google has been selling my nonfiction for me since it was invented, but it won's sell fiction and there is a big pile of money and undying author gratitude awaiting the person who comes up with the engine that will.

    • I don't know how John Locke and Amanda Hocking sell fiction, but they do.

      I think what we're finding here is that it isn't one answer; it's many

      answers. Which, if true, means that the publishers' efforts like Bookish and

      aNobii should concentrate on things that help the other conversations and

      curations happen. It would create more leverage and there's no such thing as

      a single source.


      • Lottadata

        Hocking had the advantage of getting in early and  working the Kindle discussion boards, as I understand it. She also was among the first to manipulate pricing to boost her rankings–and she did what she did back when there were far fewer books on Kindle and far fewer authors copying her techniques. I don't know the details on Locke, but I suspect he shares early adopted characteristics with Hocking.  

        Most of the strategies used by the early adopters won't work now that so many thousands more authors have turned to Kindle et al. Remember back when POD first emerged and there were a few self-published authors who promoted their POD books to mainstream success? Remember how brief that phenomenon lasted? 

        With the Invasion of the Backlists and everyone cutting prices and tweeting for dear life techniques that worked nine months ago for unknown authors are no longer going to be anywhere near as effective.

        Personally, I'm getting very tired of hearing supposed experts citing Hocking and Konrath every time the issue of web marketing comes up. I'd like to hear hundreds of cases cited, not just those two, but there probably aren't hundreds.

      • Instances of success as extreme as Hocking, Konrath, and Locke will be rare

        under any circumstances. But their stories are very different. You described

        Hocking's well, but it is also true that she worked it for *years* before

        she broke through. Locke's rise was rocket-like; I think he put up his first

        book last October and was selling hundreds of thousands by March. Konrath,

        unlike the others, started as an author published by major houses.

        These were always 1 out of X (X being a large number) occurrences. They

        always will be, but they will happen again. It is true that publishers are

        putting out their backlists, etc., but the number of readers of ebooks also

        keeps growing. And the major publishers are doing their best to keep prices

        higher than the 99 cents to $2.99 favored by the self-publishing authors.

        We are in the second inning. (Of a baseball game. In cricket they only play

        two innings…)


  • Chris

    You don't want much do you, Mike?!!  Perhaps in a few years time the net will be able to read your mind.  🙂

    But until then I'm thinking that this 'problem' may in fact be solved by tracking cookies and advertising platforms. The acquisition of customers via Google or Facebook is a science that is advancing on us rapidly. My guess is that Google's search engine and AdMob platform will only speed this targeting up. Add in FB, Twitter etc and things start to look interesting.

    At present 'they' know where you are on the net. Give it few more years and they will know exactly what it is you want as well. The ad targeting will get better than it currently is (which is already quite amazing). Once that happens it will be up to retailers, publishers and authors to decide how much they will pay to purchase a customer (and hopefully, keep them).

    Groupon spent $800m at Google last year. Rakesh Agrawal ( estimates Groupon's cost of an actual purchaser at $26.50. For Netflix the suggestion is that customer acquisition cost is around $18. Netflix is recurring subscription, so the customer converts well, I guess. Can publishers do the same? Who knows. But Amazon certainly can because they are the platform and retailer (and publisher!). But it all comes down to data mining … you just gotta give it a little more time, Mike.

    And keep up that web surfing … with tracking cookies turned on! 🙂

  • Candy Paull

    Amazon's algorithms don't yet know how to discern between fiction and non-fiction. I can't tell you how many novels are ending up in the non-fiction bestseller lists. Not a very good system if even the basic categories are wrong.

    • I sorta doubt that is a common problem. That info is from metadata

      provided by the publishers and is the same for everybody.


      • Cindy

        Sadly, you’re wrong. I know it’s hard to believe, but just ask a cataloging or metadata or collection development person from any of the big jobbers, including OverDrive. Whoever the publishers have put in charge of metadata sometimes can’t even tell the difference between fiction and nonfiction. This is especially true of backlist titles.

      • Thanks for the comment. It is indeed a sad state of affairs if the situation
        is as bad as you say it is.


  • I find books to read through review sites. I have a subsctiptiopn to a few who read the same sort of books as me, then I check them out further on Goodreads, once I know what I want to read, I just locate it – usually on Kobo. I don't browse as I would in a book store.

    You can browse on goodreads by going to category lists and because they're more specific than in the electronic bookstores that might work better for people.

    • Sounds like a sensible process. The next step is a mechanism for you to pass

      along what you're learning so the next person doesn't have to do the same



  • ClaudeNougat

    Thanks Mike, this is a very interesting post. I was intrigued by your problem of looking for the next read digitally and not finding satisfactory recommendations. I really thought (goes to show how ingenuous I am !) that Amazon for one had solved the problem of looking for ebooks. I guess I was wrong!

    Come to think of it, I have a friend who recently complained she couldn't locate what she was looking for because the recommendations her Kindle kept throwing at her  were books she knew she didn't want to read: she was looking for the “latest” thriller, and because she had loaded up her Kindle in a first burst of enthusiasm with lots of cheap-priced Indie ebooks, the Amamzon search engine seemed convinced that was what she wanted – when she was actually looking for the top titles pusblished by legacy publishers…In the end, she looked elsewhere on Internet (I don't know where) and got waht she wanted.

    So we seem to have two problems here: one, we as readers, still have to learn how to effectively use the search engines with “better” keywords closer to what we really want; two, the search systems need to be improved – I'm not sure how since so far they're based on one's purchases on sites they know about and we can always befuddle them by moving somewhere else and doing something “unexpected”.

    This is a conundrum for Bookish et al. but it may well be a window of opportunity for clever bookstores that keep an attractive physical presence coupled with online presence. Because, in the end, it's fun to roam the shelves of a good old-fashioned bookstore and find a gem by…serendipity!

    • No doubt that bookstores are the best place to shop. It won't keep a lot of

      them alive though.

      There are myriad problems making online discovery work. Amazon has powerful

      algorithms to apply to full-text searching; I doubt most other retailers

      have that. So they're relying entirely on publisher metadata to know what's

      in the book or who would be interested.

      There are the problems you cite with recommendations. The problems, of

      course, are more severe for print books because print books, unlike ebooks,

      might *frequently* be bought as gifts. My wife gets recommendations from

      Amazon all the time based on purchases she's made for one or another of our

      10 nieces and nephews. (I could be wrong on this or they might have

      corrected it, but I don't even think Amazon distinguishes between books you

      bought and shipped to yourself and books you bought and shipped to somebody


      This will all get better, iteratively, in a variety of ways. As other

      commenters have noted, the social reading sites try to pick up the slack as

      well. I just don't choose to use them. I go to the “store” and want them to

      present me good choices, just like I've always done.


      • ClaudeNougat

        You're a tough guy to please!

      • Some would say “nearly impossible.”


  • Pingback: E-böcker | rekommenderade artiklar – 27 June, 2011 | En blogg om e-böcker()

  • Jack3d

    Jack3d Review
    Jack3d Supplement
    Buy Jack3d