The Shatzkin Files

In the 1990s, Mark Bide would always begin the “Publishing in the 21st Century” conferences we ran by reviewing the research we had done around some aspect of digital change in publishing with the admonition that book publishing was “many very different businesses.” By that, Mark meant that trade publishers (who sold primarily through bookstores) were quite different from college textbook publishers and schoolbook publishers and sci-tech publishers and database publishers (who did not, and shared different dissimilarities with each other).

All of them were in the “book” business because all of them put their publishing output into bound pages for packaging and sale. But, aside from that, the commonalities in business model were all within the segments of book publishing, not across them. And when we were running these conferences 15 or 20 years ago we wanted our attendees to understand that how digital change might affect trade books might be quite different than how it would affect textbooks or professional books.

This was a continuing lesson. When O’Reilly and Pearson established Safari as a subscription database of books for programmers, it was a successful commercial play that wouldn’t have worked for a publisher of mysteries or biographies. And, indeed, the principal disruption in the trade business over the past decade has been the reduction of retail shelf space, a factor which affects non-trade publishers very little.

It has been suspected in these quarters for quite some time that the trade business was, on its own, going to demonstrate that it is actually many different businesses. That fact may now be manifesting itself in visible ways.

Last week Nate Hoffelder of The Digital Reader pointed my eyeballs at a story from the UK about a very prominent gardening author who, at age 85, has decided to stop writing gardening books because he believes his audience now gets that information from the Web, not from books.

Dr. David Hessayon created the Experts series of gardening guides and has been delivering more and more of them for over five decades, distributed in the UK by a division of Random House. But his sales figures and his insight into digital change tell him that “the how-to-do-it book has lost its absolute supremacy. To write a bestseller now you need to choose something that you can’t look up on Google.”

Hoffelder offered his take on this.

Then, entirely coincidentally, came this very much related story in Monday’s New York Times. The Times focused on the efforts, of which there are many, to create something different than a straight “conversion” for an ebook, or simply moving what was on a page to a screen. The reporter spoke to some of publishing’s leading pioneers around that problem. The confusion, in the industry and in this piece, is that the pioneers aren’t tackling the same problem. Peter Brantley, a library-rooted digital pioneer identified for his role organizing the Books in Browsers conference, talks about the limitations of the printed book in constraining how stories can be told. I am skeptical about what productive results can come from pursuing that possible opportunity. My sentiments are much closer to what was expressed by Peter Meyers of Citia, who said “a lot of these solutions were born out of a programmer’s ability to do something rather than the reader’s enthusiasm for things they need. We pursued distractions and called them enhancements.”

(I worked with Pete Meyers on a project a few years ago and some useful videos resulted.)

That said, it is no surprise that the program from Citia is highly practical, breaking complex non-fiction books into “cards” representing the ideas inside the book. Inkling has used a similar approach to make ebooks from how-to books, including creating an online bookstore from which to sell them. (Inkling has also made the point that the “card” paradigm also makes the content more discoverable, by making the cards themselves searchable and discoverable.) The “how-to” ebookstore is definitely an idea on the right track, but it will take a while to build enough awareness and traffic to find out whether the ebooks will sell in sufficient numbers for people to make money.

The books Citia applies its thinking to — idea-oriented books like Kevin Kelly’s “What Technology Wants” — are quite different from the how-to crafts and photography and cooking books Inkling is featuring. And they’re miles from novels, maybe light years from the more inventive replacements for the print novel that Peter Brantley is thinking about.

The Times piece focuses on the fact that the attempts to “change” the digital version of the book from what the printed version was — with interactivity or social or visual elements — have universally failed commercially. This is true. The piece Nate Hoffelder was inspired to write poses a more useful query than whether publishers can invent new forms that will work commercially: “Is the Internet a Greater Threat to Publishers than Self-Pub eBooks?”

But neither gets to the extension of the point Mark Bide made repeatedly two decades ago. Now it is the trade book business which is showing it is many book businesses, a fact that is being revealed by the shift to digital. And publishers are increasingly realizing the truth of this and that they have to focus on that fact as they plan their futures.

Here’s the simple fact that none of these three articles say. We have proven beyond any reasonable doubt that digital versions of narrative immersive reading — which I define as books you read from page one to page last — if made reflowable will satisfy the vast majority of the book’s print audience. Some people have switched to devices and some haven’t. Some stubbornly prefer printed books. Some find reading on a phone too cramped or reading on a computer too confining. But almost everybody finds reading on an ereader to be quite satisfactory (even if they don’t find it preferable to print). And if the book reflows and you can pick your type size, the ways it could have been improved but wasn’t always (seamless note-taking ability, improved navigation, ability to share) don’t interfere with your personal reading enjoyment. So these books have “worked” commercially as ebooks, particularly since the cost of getting to a digital version is trivial.

However, the complementary fact is that we have not yet found a formula that works for any other kind of book. (And with all due respect to Philip Jones of The Bookseller, whose piece on this subject is much more “on point” than the other three, pointing as he does to what Pottermore has done and can do is hardly a prototype for a dedicated book publisher.) How-to books haven’t sold well as ebooks. Reference books haven’t sold well as ebooks. Cookbooks haven’t sold well as ebooks. If you dip in and out; if you rely on illustrations (which maybe should be videos); if your book is just filled with pretty pictures; then there is no formula for a digital version that has demonstrated mass commercial appeal. There have been successes, but they seem to be novelties (e.g. Touch Press) or on a much smaller scale than would warrant major publishers getting into this business (e.g. a small art press like MAPP Editions can claim success with 1,000 copies sold).

And even though companies like Inkling and Aptara and Aerbook are doing their best to make the process cheaper and easier, making an ebook of a complex book is going to cost more and take more creative bandwidth and, in some cases, entirely new skillsets from the publisher (and perhaps the author) than the conversion of a novel. A complementary challenge is how these books translate to online sales. Narrative fiction and non-fiction sells well online, whether in print or digital form (so, those “stubborn” print readers are still satisfied). It’s a heavier lift to sell print illustrated how-to, art, and reference books online.

What this means is that the digital future for narrative reading — fiction and non-fiction — is much clearer than it is for any other kind of book. Publishers of novels can apparently count on their sales shifting from print to digital and from in-store to online without losing a lot of readers. And with not much in the way of conversion costs, publishers of these books can proceed with their development with some confidence that the changes in publishing’s landscape and ecosystem won’t throw the calculations they are making for future profits on today’s acquisitions into a cocked hat.

But publishers of everything else have no basis for similar confidence.

No general publisher that I’m aware of has announced “we won’t do illustrated books anymore”. I have purely anecdotal evidence from people who once worked there and left that Random House — the one publisher I know that really tried to convert a lot of its illustrated content to ebooks over the past few years — is de-emphasizing illustrated book publishing. I have been given to understand that one of the leading art book publishers is now doing more straight text publishing, which is sensible if art books don’t port to digital.

As for Dr. Hessayon, I know what I’d suggest if he were my consulting client. With digital content about gardening that has been being created since 1958, the chances are very good that he has a database of information that could constitute a whole new resource for gardeners in the 21st century. Perhaps there is a publisher who can do something with that, but it is perhaps more likely that a producer of seeds or fertilizer or a garden center retailer would have just read an article on the Internet about “content marketing” and see Hessayon’s last half-century of work as a great jumping off point for a new offering for the next half-century. The good doctor is right that “books” are no longer the best commercial form for monetizing a lot of information, but that doesn’t mean the information isn’t valuable, if it is delivered in different sized chunks under a different commercial model.

It would certainly appear from his experience that he’s concluded that the publishers’ distribution network no longer fits his content and its presentation. Unfortunately for today’s publishing incumbents, there are other skills that are required to be a good book publisher which also may no longer have commercial relevance for that content. So the question for publishers is whether their skills and assets are right for whatever will be the new way to present this kind of content. The answer — except for long-form reading — is not self-evident.

But, of course, publishers of illustrated and other complex books have to keep trying to find a solution that works and the only way to do that is to keep creating new digital products out of their books. A panel of people who can help them do that effectively and efficiently — Pavan Arora of Aptara, Gus Gostyla of Inkling, Ron Martinez of Aerbook, and Bill Kasdorf of Apex Covantage — will discuss the topic “Crossing the Chasm: Finding Digital Solutions for Non-Narrative Content”, moderated by industry veteran David Wilk at Digital Book World on January 14.

As part of the promotion of the Digital Book World conference, I do some interviewing with the very capable Jeremy Greenfield, the editor of their blog. And Jeremy takes our conversations and chops them up into short pieces around the themes of our show. Since the focus of Digital Book World is “how digital is changing publishing”, Amazon is a topic of great interest and one we try to address in an original and enlightening way.

In my interview with Jeremy, for which he published very brief but entirely accurate excerpts, I did say that publishers would face a real selling job with authors when Amazon’s share grows by another 25% from its current base or if Barnes & Noble closed. Neither of those things is likely to happen in the next few years. If and when the day comes that one of those things does happen, not all publishers would be entirely defenseless even with today’s arsenal of capabilities. And Jeremy’s piece closes with my suggestion that publishers can help themselves by doing “digital marketing at scale, which is audience-centric in its thinking.”

Despite how this is interpreted in some circles, it does not add up to publishing “spiraling down the drain”.

Amazon is already truly disruptive and it isn’t clear to anybody but those on the inside of Amazon exactly how disruptive. I’ve written earlier that we know nothing about the used book marketplace they host and foster, which we must assume cuts into sales, particularly of bestselling books which have many copies in circulation. A recent discussion on a mailing list I’m on revolved around what we don’t know about how many ebooks are being published. Why? Because Bowker, which issues ISBN numbers and therefore helps us count the titles going into the marketplace, doesn’t necessarily get to touch (and count) titles that stay entirely inside of Amazon and therefore only use the Amazon “ASIN” substitute for the ISBN. Other ebook retailers will handle titles without ISBN numbers, but only Amazon has a large enough market by itself to make a substantial number of self-publishers work with them alone.

And now we have the anomaly of sales reporting from the AAP, once again working without totally internal Amazon IP, that suggests ebook sales are going down. Are they going down? Or are self-published titles exclusively inside Amazon taking share away from the part of the business we can see and count for ourselves and masking the ebook sales growth that is actually taking place? I have no evidence, but that strikes me as a more likely reality than that ebook sales have actually fallen year-to-year recently.

What that means is that we are developing two publishing businesses. One of them includes all of us: all the publishers, all the retailers, all the industry bodies counting books and sales. And one of them is “private” or “proprietary”; it is Amazon. They are publishing an unknown number of titles selling an unknown number of copies netting an unknown number of dollars under a numbering system nobody else can crack or track.

Actually, Amazon is not entirely alone in wanting proprietary titles. Perhaps there are some within Nook or Kobo, but hosting proprietary titles to establish themselves in the market is the declared strategy of upstart retailer Zola Books. Last week they announced exclusive titles from Joan Didion and her late husband, John Gregory Dunne. They think having showcase titles of this kind will enable them to crack the ranks of established ebook retailers. I think it would take a lot more of them than they’ll ever get to make a dent, but time will tell. And if they don’t sell a lot of the ones they have, it will become impossible to persuade anybody else to give them such an exclusive on any basis.

But Amazon, being more than half the market already for a lot of genre fiction, can use painless (to them) financial incentives to induce authors to give them exclusives through the KDP Select program. So they get them in numbers none of the rest of us can count but which could conceivably be large enough to actually make industry figures inaccurate.

My assumption is that Amazon can do more for a book inside Amazon than a publisher or author can working Amazon from the outside, all other things being equal (although the U-turn from the ambitious Larry Kirshbaum publishing program might cast doubt on that). And the publisher takes a big share of the Amazon-generated revenue. That means that the publishers have to make up the difference in revenue for the author in one or both of two ways:

They have to do a superior job publishing the book — editing, positioning it in the marketplace, selling rights, and sustaining a marketing effort that will be largely digital — so that it sells more even inside Amazon than it would without those efforts. In other words, they have to assure that “all other things” do not remain equal.

They have to sell lots of books outside of Amazon so that the revenue from the larger publishing ecosystem makes up for the Amazon-generated revenue that the author shares with the publisher.

The shift that has taken place so far is apparently not crippling publishers at all. There are no clear tallies about this, but it certainly feels like there are more authors moving from self-publishing to a publishing house (to borrow a term that usually has a different meaning in our business: “discovered” by publishers because of their self-publishing success) than the other way. So either they’re able to make more money, or they really appreciate the full bundle of editing and marketing services a publisher provides, or they value the broader exposure through a publisher’s entire distribution network more than the perhaps-higher revenue they could make from fewer sales through Amazon alone, or some combination of the three.

My point, and what should be a broad industry concern, is that the publisher’s challenge continues to get steeper. Amazon’s share is growing in relation to the rest of the market and more and more service offerings for editing and marketing are making it ever-easier for authors to entertain a non-publisher option. There is a very small but growing population of authors with lengthy backlists who have gotten their rights back, or secured their ebook rights alone, and are able to consider alternative paths to market.

Although she wasn’t the first, Jane Friedman saw this very early — and it is the opportunity that got things started for her Open Road Integrated Media, probably the largest new publisher built during our current shifting paradigm. Richard Curtis of E-Reads and Arthur Klebanoff of Rosetta were pursuing a similar strategy before Friedman got started, but she found the funding and added the promotional sizzle to build a bigger business faster. (It is still an open question whether the companies that are building themselves by offering more generous royalty splits for already-established backlist have a sustainable business model.)

We’ve said repeatedly in this space that the publisher’s time-honored core proposition has been “we put books on shelves”. That is changing and the new proposition has to be “we will help authors reach their whole audience”. A very smart executive from a major house suggested another formulation that makes sense: “publishers are experts at building author brands.”

Either of those, as a competitive statement against Amazon, will almost certainly reflect a potential advantage for authors. But as the difference between what is Amazon’s audience and what is the whole audience gets smaller, the publishers’ challenge gets harder. And only by doing a smashing job at both publishing in a way that sells more on Amazon and by maximizing the market outside Amazon will publishers retain their power to attract authors in the years to come.

The answers for publishers as seen from here are “verticality”, or “audience-centricity”, combined with scaled skills (and tools) to do digital marketing in ways the authors can’t on their own and which Amazon isn’t likely to develop. The two go together: focusing on an audience enables a publisher to build scaled capabilities to reach that audience that others without that focus will not have.

There have always been publishers that have gone “down the drain” or, more likely, seen themselves become part of some other publisher rather than a stand-alone entity. We will certainly see consolidation in various segments of the industry at the same time that we will see lots of new smaller entrants attracted by book publishing’s diminishing cost of entry. (We call this atomization.) But seeing that things will get harder is not the same as seeing a pending apocalypse, and recognizing there are benchmarks that would signal a real escalation of the challenge is not the same as saying we’re about to hit them.

The topics covered in this post will get a thorough airing at the Digital Book World conference on January 14-15, 2014. (Here’s the full program.) Our Amazon coverage will include presentations from Brad Stone, Benedict Evans, and Joe Esposito, followed by a panel discussion among them. Professor Dana Beth Weinberg combines her data analysis skills as a sociologist with her publishing interest and knowledge as a romance writer to present a unique perspective on the changing dynamic between publishers and authors. And Phil Sexton, the publisher of Writer’s Digest, will present the results of his organization’s survey of more than 5,000 freelance writers, capturing an up-to-date picture of how writers view the choice between working with a publisher and putting their material out on their own.

We are in another high-funding era for digital start-ups. The book business has always looked ripe for disruption, but never any more so than now. With bookstore shelf space shrinking, ebooks growing in very uneven ways across the types of books that are published, and everything about technology getting cheaper, everything is up for grabs.

It is not a new thing that the world looks different to the companies funded by the revenues from the legacy business than it does to outsiders, some of whom want to bring tech disruption into collision with the legacy business.

Publishers see an ebook business that has been very commercially unkind to the digital versions of books that aren’t immersive narratives. Start-ups and their funders see publishers too stuck in old forms, and unable to break away from a book-style presentation when the content and use cases would call for something quite different.

Publishers see a printed book marketplace that is dominated by Amazon with less and less room for books in stores. Start-ups and their funders see an opportunity to gain further digital discovery by making the content easier for people, and web crawlers, to “see” online. And they also see making digital versions of books easier to “share” as an aid to discovery; publishers often see it as an enabler of unauthorized distribution that could cut into sales.

Publishers see books as products driven primarily by interest in the author or genre (for fiction) or the subject (for non-fiction). Start-ups and their funders see reading as an activity at least partly driven by convenience and availability and the ability to share the reading experience.

Publishers see Netflix and Spotify and think, “How many people read more than a book a month? The subscription model doesn’t really apply to our business.” Start-ups and their funders see that the consumers of all other content really like the subscription model and they can’t see why it wouldn’t work in the book business, too.

So we have, for example, several serious initiatives around subscriptions: dedicated (and often well-funded) start-ups like Oyster, eReatah, Skoobe and 24 Symbols, as well as initiatives from the totally-established Amazon.com and the differently-established Scribd. At the same time, some agents are outspoken in their objection to the whole concept, seeing it as a way that commercial power will pass from the author brand to the subscription brand. Publishers generally pay close attention to what agents say. Whatever the reasons, as of this writing only HarperCollins has broken ranks among the Big Five to place any substantial number of books in subscription services.

If you get many of the start-ups to speak candidly about publishers, they’ll often accuse them of being hidebound, unimaginative, wedded to old ways and models, and still “experimenting” with things that should be well-established.

If you get many of the publishers to speak candidly about start-ups, they’ll bemoan the fact that they too often don’t understand how the business really works or the true commercial imperatives at the publishing houses, which must continue to sign up and please authors and harvest revenues that still come overwhelmingly from sales of one item at a time to one consumer at a time through intermediaries.

At Digital Book World in January, we have five elements in the program to address the relationship between start-ups and established publishers.

First: we are running a survey of start-ups and publishers to get them each to talk about what they expect from the other. If you work for a start-up or your job at a publisher includes meeting with and evaluating start-ups, please respond to the survey! We will announce the results at DBW.

Second: Ron Martinez, who has a start-up (Aerbook), partly financed and supported by an industry leader (Ingram) and a long background in tech, patents, and design, will speak about the relationship between start-ups and incumbents.

Third, Fourth, and Fifth will be three panels exploring the question from three sides.

A panel of start-ups, which will include Martinez and Andrew Rhomberg of Jellybooks and two others we’ll pick after we see the survey results, will talk about what it takes to get traction with publishers, what publishing, marketing, or ecosystem problem they’re addressing, and explain their own vision of a path to success for their enterprise.

A panel of publishing business development people, including Rick Joyce of Perseus Books Group and Leslie Hulse of HarperCollins, will talk about how they view start-ups. What makes them give start-ups a meeting? What makes them engage? How much buy-in do they need from the rest of their company to be able to work together?

Finally, a panel of investors in start-ups, three of which are owned or controlled by existing publishing entities (Ingram, Macmillan, and Harvard Common Press) will talk about what persuades them to fund a start-up and what disruption they see on the horizon for publishing from the start-up community.

Very good publishing minds from three continents around the world, including Arthur Attwell,  Javier Celaya, and Brian O’Leary, have expressed themselves recently on this very problem. Although I disagree with chunks of what each of them has to say (as Jeremy Greenfield’s interview with me on the DBW blog makes clear), they individually and collectively express the real challenge of finding both workable paths to the future and workable ways for innovators to work with incumbents to get there.

The post from Jeremy triggered an exchange on Twitter among Rhomberg (from whom it inspired a thoughtful post), Peter Turner, and me which surfaced another important point. An incumbent’s job is to continue to maintain economic viability. A start-up’s objective, often, is to “change the paradigm”. If the paradigm does change, the incumbent needs to roll with that, but they don’t need to be an instrument of change. A start-up often does. That is an inherent difference in perspective that a start-up can’t afford to ignore.

As a guy who questioned why anybody would want another device just to read books when Amazon introduced the Kindle, I’m the first to admit that predicting in advance how an innovation will do — including the observations I made with such conviction in the DBW piece — is rarely a slam dunk.

It isn’t likely that our sessions at DBW will help anybody predict which innovations will succeed in the future, but it might help both start-ups and incumbents develop more mutually productive approaches to engaging with each other. That’s certainly the intention.

Don’t forget to respond to the survey if you are either a start-up or in a role at a publisher that involves meeting with or evaluating them. We’ll be collecting responses through next Monday, November 18.

The industry got the news that Amazon was probably reassessing its own publishing program a couple of weeks ago when it was announced that Laurence Kirshbaum was stepping down as the head of Amazon Publishing and being replaced by a 14-year veteran of the Seattle company, Daphne Durham. Whatever are Durham’s strengths and connections, they don’t include the familiarity with the New York publishing scene and agents that Kirshbaum brought.

While this certainly does not suggest an overall reduction in Amazon’s publishing activity, it does signal a change in tactics. It would appear that the unorganized but united stand by Barnes & Noble and independent bookstores to boycott Amazon-published titles and refuse to give them shelf space made it virtually impossible for Amazon’s publishing enterprise to compete with the big houses for brand name authors. The few that they tried — Penny Marshall and Timothy Ferriss wrote the high-profile titles that were watched — had disappointing results. Whether that was largely because the stores wouldn’t play along or for other reasons (not all books by famous authors or celebrities are equally edited or equally appealing), the overall environment did not leave agents or the authors everybody wants panting for an Amazon publishing deal.

Retreats — apparent or real — by Amazon are rare. (The last one we can recall is when they pulled the buy buttons from Macmillan titles in 2010 to protest agency pricing and very quickly rescinded the action.) But it would be a mistake to think either that Amazon is less interested in publishing than they were before or that the threat they pose to publishers’ relationships with authors is no longer something publishers need to concern themselves with.

In fact, all the recent evidence suggests that Amazon’s market share is still rising. The Bowker numbers reported at the end of July of 2012, trying to measure who got the Borders sales (which were 10% of the total when the retailer went out of business) put Amazon’s total share of the book market at 29%, up from 23% a year earlier. In that same report, it was reported that B&N had gained a point of share, up from 19% to 20%. So Amazon out-benefited B&N from Borders’ collapse by six to one.

Earlier this year, it was reported in Britain that Amazon had a whopping 79% of the burgeoning ebook market. That’s more than they have in the US. It is also apparently the case that Amazon has the lion’s share of the online book sales market in the UK (and, along with their subsidiary company The Book Depository, most of Europe and the English-speaking world).

The share of total sales that goes through their registers is only one measure of Amazon’s disruptive growth. They’re also signing up more and more books directly to their imprints (the genre publishing growth continues unabated and was never heavily dependent on Kirshbaum) and getting more and more books through authors self-publishing. And as they disintermediate publishers by bringing in books directly by either means, they also threaten their competitive retailers in all venues. Although you can be self-published through Amazon and continue to distribute to other channels, they offer financial incentives to discourage that.

In fact, Hugh Howey, the enormously successful self-publisher of “Wool”, told us a year ago that the decision to broaden his distribution base to include Nook and other platforms cost him money. He did it because he thought it was the fan-friendly thing to do but he’d have made more money on his ebook sales if he’d sold fewer units and given up the other formats.

(KDP Select is the program that demands exclusivity. By enrolling, authors get their works in the Kindle Owners’ Lending Library, increased royalties on sales outside the US, and access to additional promotional tools. You can still have your book on sale in physical, “or in any format other than digital”.)

We see Amazon growing into a large and slightly separate book industry of its own. They don’t use the book business’s standard ebook format, epub; they use their own format, mobi. (The Amazon “flavor” is AZW, and they also have the newer KF8.) They don’t care much whether a book has an industry standard ID, the ISBN number. Amazon assigns its own number, unless the publisher has a 10-digit ISBN they can use, which they call an ASIN. They own a must-optimize author page (Amazon’s author page affects an author’s discoverability on Google; the converse is not true) and a must-use book readers’ social network (GoodReads). They have their own print-on-demand operation making it simple for an author to set up both ebooks and print at the same time.

The “advantage” a publisher has pursuing authors is that they can offer a much broader distribution base as well as their honed skill at marketing and publicity. But there’s a price for that; self-publishing with Amazon brings an author four times the revenue for ebooks and somewhat more for every print copy sold as well. Whether Amazon is a quarter, a third, a half, or more of a book’s sale depends on the book, but authors will be increasingly facing the choice Hugh Howey faced: publish exclusively with Amazon and sell a bit less but make a bit more, or publish to a broader audience through a publisher (or on your own) and make less money. Apparently, many authors are doing 90-day runs of KDP Select to get a boost at Amazon, then switching back to broader distribution

Fortunately for the rest of the publishing business, the shift to ebooks and to online purchasing may have stalled. In the US, Amazon appears to have about 60-70% of the ebook business, and ebooks constitute about 30% of the total business. But the ebook share is much higher for immersive reading, higher still for fiction. For fiction, more than half the sales of many titles can be digital. And the print sales are anywhere from 25% to 35% online. So for fiction, Amazon may already be nearing half the total sales for many titles.

We wouldn’t expect the slowdown of the shift in sales to last. New offerings of ever-cheaper and more-flexible devices, more and more cheap ebooks in the market (discounting the backlist ebooks seems to be publishing’s latest most common marketing trick), and the natural growth in digital interaction as older people exit and younger people with new credit cards replace them, pretty much assure that the online sale will continue to grow in relation to the store sale. As that happens, as the 2012 measurements after the demise of Borders showed, Amazon experiences organic growth.

So, when does Amazon’s share growth stop? And who is left standing when it does? Here we have to enter a realm of pure speculation; there are no data points that can help us figure this out.

To answer these questions, we need to look at the book business in segments.

For narrative text, books that one reads from the first page to the last, we’d expect continuing growth of digital. For genre fiction (including YA), which has the additional characteristic of having audiences that consume many titles a year, we’d expect a lion’s share digital market — 80 percent or more — to be common within a couple of years. For those books, Amazon will continue to just eat away at the publishers’ position. More and more of the genre readers will migrate to them because they’ll have an increasing number of titles on an exclusive basis, more — and more aggressive – price promotion, and probably a variety of subscription opportunities. That should lead inexorably to more and more of the genre authors being willing to publish with them exclusively because they’ll be able to reach an increasingly large percentage of the reader base through digital and Amazon alone.

If I were looking for the first candidates not to be “left standing”, we’d expect to find them in genre publishing. In time, the big publishers will increasingly focus on “big” genre titles, rather than lengthy genre lists.

I also expect more DRM-free trials, particularly in genre fiction, so that publishers and third-parties can sell mobi files to existing Kindle customers. For while genres are where Amazon has their greatest potential strength, it is also true that genres are where publishers have the best chance at building brands and direct customer relationships that matter.

More general fiction and non-fiction will be read mostly in digital form in a short time too, although the hardcovers for those books will continue to exist. But for the big players in general trade, there’s another problem besides Amazon to deal with. That’s the new publishing behemoth: Penguin Random House. I would guess (all we can do) that by three or four years from now, the first choice for most authors will be either PRH or Amazon. PRH will provide the biggest reach; Amazon will often provide the biggest potential revenue. The other general trade houses will fight each other for the authors that don’t want to be part of either behemoth.

For illustrated books and children’s books, the environment will be different. Stores will remain important, but there will be fewer of them (and therefore fewer books of this kind published). The bookstore I’d imagine in several years will have far more illustrated and gift books in it as a percentage of the total title mix than it does now.

What I think will save publishers from disappearing, oddly enough, will be a loss of interest at Amazon in taking more market share. This conclusion comes from a combination of something I learned from people at Google about Google and what is clear from Stone’s book.

Last spring, I visited a Google installation that was not about the book business, but about an online game. The game is a big online experiment in engagement. Googlers showing us around were thinking about the revenue potential of the game, which was not supposed to be their primary concern. They had come to the conclusion that $100 million in annual revenue would be achievable, but they didn’t think they’d be able to go after it. Why? Because nobody in a responsible position at Google would take ownership of something as small as $100 million in revenue.

Brad Stone paints a picture in “The Everything Store” of Amazon as, above all, a highly rational company. Jeff Bezos can be impetuous, but he’s not nuts. He is zealous about the things he cares about because he believes they matter: customer happiness being number one on the list. As the book business becomes a smaller and smaller part of the total Amazon picture and the challenges that matter to the business revolve around delivering your fresh produce in 30 minutes, not 90, it is likely that Amazon will have less and less interest in squeezing just a little bit more margin out of the book business. There will be easier places and easier ways to make money.

Amazon achieved the position it has in the book ecosystem through a combination of brilliance, execution, natural forces, and some good luck but, above all, focus. It had to take some big chances with pricing and margin to get where it has gotten, but that’s not really necessary anymore. Doing some very logical and natural things, like the new Matchbook program and rolling out more subscription and pricing offerings (like their new “Countdown Clock” discounts for new Kindle titles) will keep their share growing and their competitors scrambling. They will also almost certainly be coming after publishers for more margin (as will their equally dominant counterparts on the store side, Barnes & Noble), but it would seem unlikely that they’ll see the need to extend themselves to sign up authors or build out their ability to distribute print to other people’s stores.

Amazon will certainly continue to make it difficult for publishers to use price offers as a way of teasing away some of the direct ebook business. Publishers are finding that increasingly tempting as more and more vendors emerge who can solve the tech challenges for them. But even with publishers taking some ebook share directly, and more of them will, chances are that the ebook business will grow faster than the publishers’ shares and that Amazon’s growth, partly at the expense of other ecosystems, will not stop.

So the good news for publishers is that the business they now have will look less and less appealing compared to other worlds Amazon might conquer. That should save them from having a bulls-eye on their backs, but it will remain a very challenging environment where their biggest customer is the most powerful force in the marketplace and growth outside that customer is harder and harder to achieve. The publishing activities of Amazon will continue to get bigger; the industry of other publishers will continue to get smaller. But we are probably in for a period of slow and steady shifts rather than cataclysms.

As long as Barnes & Noble can stay healthy and the other ebook platforms aren’t crushed by losing titles to Kindle exclusives, that will remain the case. And that means “for quite a while” but not “forever”.

Remember that Brad Stone will be joined onstage by analyst Benedict Evans and publishing sage Joseph J. Esposito for a wide-ranging discussion about Amazon at Digital Book World in January.

Note that I also posted on Amazon yesterday. That piece describes three important pieces of their story that didn’t make it into Stone’s book.

And, if you’re from a start-up or your job at a publisher includes meeting with and evaluating start-ups, we really want your response to our survey, which will inform our dialogue about start-ups at Digital Book World.

The publication of Brad Stone’s book about Amazon, “The Everything Store”, is the catalyst for a lot of new discussion about the topic most difficult for the book business to discuss. It is pretty much impossible to be in the book business without benefiting from Amazon’s market reach. But it is also pretty standard fare to be worried about what the impact will be on your business as that market reach grows.

Amazon is, at the same time, both the biggest customer for most publishers and many wholesalers and their most potent competitor. They compete with every bookseller for sales, which weakens the brick-and-mortar trade, and thus dilutes the core value proposition that publishers have always offered: putting their authors’ books on bookstore shelves. Weakening the diverse bookstore ecosystem weakens the wholesalers. At the same time, Amazon competes with publishers for authors, both through their publishing programs and through their self-publishing services.

They also compete with the free ebook lending from public libraries and the various ebook subscription services with their Kindle Owners Lending Library, a service offered to their Amazon Prime customers that makes a large number of titles — many published by Amazon or self-published exclusively through Amazon — available for no-additional-payment downloads.

And they are capable of creating propositions that every other retailer would love to match but would find quite difficult to do, such as their recently announced “Matchbook” program which offers a free or very cheap ebook edition to any customer who has bought the print version of that book from Amazon. In fact, many publishers believe in the print-and-digital “bundle” and have made efforts to engineer it for bookstores, but it is hard to do that cost-effectively. It isn’t hard for Amazon.

Candid public conversation about Amazon from other players in the industry is pretty much a non-starter. Every publisher is walking the fine line of trying to make their sales grow through their largest account and, at the same time, somehow growing their sales faster everywhere else.

And that’s just about impossible. For the few years (just concluded) when all ebook sales were growing, publishers were seeing upswings in their business with other digital accounts besides Amazon. But recent evidence suggests that ebooks have hit either a point of serious resistance or a temporary plateau so even that may not be true anymore. It is likely that for many publishers Amazon represents the only significant account that continues to grow.

Last week, Jeremy Greenfield of Digital Book World interviewed me about “why it is so hard to compete with Amazon”. Since this is a topic of such widespread interest but also so hard for so many of the industry leaders to discuss, extending that discussion seemed warranted.

In this post, I want to cite three important aspects of Amazon’s history — important as far as the book business is concerned, although not necessarily to the overall picture Stone successfully conveys of the Bezos vision and the strategy and culture that achieve it — that didn’t make it into Brad Stone’s excellent book. In a subsequent one, I will explore what I think are the two key questions about Amazon that everybody in the book business is quietly asking:

* When does Amazon’s share growth stop?

* Who is left standing when it does?

About those two questions, all we’ll say here is that Stone’s book gave me fresh insight into the possible answers.

Now for those three missing points and why they’re important.

I first raised these questions and wrote about Amazon’s squashing of Ingram Internet Support Services (known as I2S2) about two years ago, but what I think is a very important story didn’t make “The Everything Store”.

As Stone describes clearly, Amazon began its business basically standing on Ingram’s shoulders. They stationed themselves in Seattle, near a big Ingram warehouse in Roseburg, OR. When Amazon started, they were able to take a customer’s order and money; order and receive the book from Ingram and deliver it to the customer, and then sit on the cash for a while before they had to pay Ingram for the book.

Pretty early in the piece, Ingram saw that all retailers could take advantage of this capability of theirs. So they created the I2S2 offering and went out to book retailers to persuade them to use it the same way Amazon did. Of course, at that time Internet retailing of books was a tiny part of the market, but Ingram hoped that the opportunity to offer a cash-flow-positive service to their customers would entice some stores, who were already Ingram customers, to diversify the choices for online customers.

Before I2S2 could get off the ground, Amazon killed it with high-profile discounting to as much as 40% off the cover price, effectively taking the profit out of Internet sales. This move was seen as a tactic to grow the customer base quickly and satisfy the investment community’s desire to see growth in top line and in customer base. That’s accurate. But it also stopped what could have been serious competition in its tracks. Booksellers profiting from their stores had little patience to build online business that was small and would now not even be profitable.

A publishing executive who was at Random House in the late 1990s recalled in a conversation we had last week that Peter Olson, who ran Random House at that time, told him not to worry about Amazon because their share grew by about 1% per year. In fact, that’s probably just reflecting that the consumer tendency to purchase online grew by 1% per year. The executive who told me this story made the accurate point that Olson was proven right about the share growth over many years, with additional surges when events like Borders’ closing took place. (And, of course, he told the story because we both knew that Olson was proven wrong that this 1 percent growth a year was nothing to worry about. “When does it stop…?”)

But imagine if Amazon had not reacted to the existential threat of a multitude of potential competitors by trading their margin for survival!

The I2S2 experience of the late 1990s adds some poignancy to a piece of excellent reporting by Stone about a meeting Amazon had with Ingram early in the century when Amazon’s stock was falling and some industry players were worried about whether they could pay their bills. Stone reports John Ingram making it clear to Amazon that Ingram could not afford an Amazon bankruptcy. Clearly, Ingram’s credit policies had continued to fuel Amazon’s growth in the years that had elapsed when they killed I2S2 with discounting.

The second point that is somewhat more significant than I think Stone portrays it was Amazon’s purchase of the ebook technology Mobipocket in 2005. In those days before there was a real ebook business and an “epub” standard (which Amazon eschews, which is another story not thoroughly enough explored), the two leading reflowable ebook standards were controlled by Palm Digital and Microsoft. Palm’s strategy was to sell the ebooks themselves through sites they owned or controlled. Microsoft was going for the broader play and enabling retailers to sell their format.

But the problem was that the lion’s share of the tiny ebook market read Palm, not Microsoft’s Dot Lit format. So the retailers, one of which was Barnes & Noble, were really hobbled. They could only sell the ebook format nobody wanted. Mobipocket’s format would work with both the Palm reader and the Dot Lit reader, so selling that format would reach most of the hand-held devices then used for ebook reading. If B&N or Borders or anybody else had made a strong push for the ebook customers using Mobi, and capitalizing on the format’s ability to serve the entire ebook market of the time, the effort might have gained a foothold. After Amazon bought Mobipocket, they did nothing with it for three years until they used it as the ebook format for the Kindle. (By that time, Dot Lit was about dead and Palm’s core business in hand-held PDAs was about to be demolished by the iPhone.) Did Amazon buy Mobi to postpone the ebook revolution until they were ready to lead it? It would certainly seem that way.

The other significant item that I think “The Everything Store” underplays is Amazon’s enabling of the used book business online. Although this is a “marketplace” function — Amazon is not the seller of the used editions, independent players are (presumably, although questions have been raised about whether all the marketplace sellers are actually entirely independent) — it was Amazon’s decision to place the used book availability and pricing right on the same page which sells all the editions from the publisher. That means that everybody who searches Amazon for a title is shown the used copies that are available competing with what the publisher offers.

What is the impact of this ubiquitous used book availability competing with new copy sales at the world’s biggest book retailer? Well, actually, nobody really knows. In 2006, Amazon (for some unexplained reason) participated in a study and industry conversation about used book sales. They haven’t done it again between then and now, and since Amazon’s marketplace almost certainly sells the lion’s share of used books, there’s not much point to examining this question without their participation.

We launched a DBW survey today on “start-ups” about which we’ll write more in a future post. But if you are either part of a start-up or in the business development function of a publisher that includes meeting with them, you will find our survey of interest (and we will value your response). You can read more about the survey here or just jump in and start answering questions.

And, of course, Brad Stone, the author of “The Everything Store”, will be one of three great speakers we’ll have talking about Amazon at Digital Book World in January. He’ll be joined by Benedict Evans of Enders Analysis, who has a paradigm for analyzing Amazon as a business that is uniquely insightful, and by Joseph J. Esposito, an industry veteran with a strong background in scholarly publishing who has noticed for years that Amazon is a significant competitor in the institutional market (schools and libraries).

The new opportunity to publish a book without printed inventory has been popularized primarily by self-publishing authors and by new fledgling publishing enterprises like Entangled and Byliner following in the footsteps of earlier pioneers like eReads and Ellora’s Cave and, more recently, Open Road. This changes the economics of publishing substantially, taking a very large part of the risk out of it and decreasing a publisher’s dependence on hundreds of stores to individually agree to commit their own capital resources to display printed copies.

There have been some experiments with no-inventory publishing from the major houses, all in genres. Last year, HarperCollins launched HarperTeen Impulse and Random House launched the digital-only imprints Loveswept, Hydra, Alibi, and Flirt. And Harlequin had preceded them early in 2009 with Carina Press. In fact, these digital innovations have already given rise to what I believe is premature concern from some agents that publishers will carry this “no printing” thing too far!

Much of the promised innovation has been around publishing shorter works, works that might not have lent themselves to printed versions anyway, but they also were a way to reach out to self-published authors. Now Italy’s RCS Libri has come up with a really imaginative use of no-inventory publishing — also in a genre — as a way to test not only the appeal of a new author’s work but also the ability of fledgling authors to promote it. The concept appears to have succeeded commercially on its first attempt; it will be interesting to see whether it can be replicated by RCS Libri and by other publishers in other countries.

RCS Libri  set up a new publishing arm called “Rizzoli Lab”, a new imprint dedicated to experiments in digital. For the first effort of Rizzoli Lab, they came up with a really nifty idea. It is a series of books called You Crime, by which RCS Libri is creating a new kind of collaboration they call “co-publishing”, by which they mean that they are combining the efforts of a publisher with the efforts authors provide as self-publishers.

You Crime has four published ebooks from Rizzoli Lab, each with four short crime stories within. Four of the sixteen stories, one in each book, are written by well known Italian crime writers. The other three stories in each book are by fledgling writers, whom the Rizzoli editors found by looking at submissions but then examining the authors’ presences on the Internet. They obviously have huge numbers of people who wish to publish with them. In addition to judging the writing quality of submissions and limiting to one genre (only crime: no romance, no fantasy), they tried to evaluate the authors’ attitude toward digital and their past experience with self-publishing. They refer to what they did as “digital editorial selection”. Since RCS Libri is investing in the entire initiative (and marketing of the series, but not author marketing) they wanted to be sure they had good content to offer to the readers and strong marketing efforts to let them know it was there. Of course, their editors knew how to judge quality content. What was new was the evaluation of the fledgling authors’ digital marketing potential.

According to Marcello Vena, the digital head at RCS Libri under whose leadership this has all happened, the established authors participated in the project at least partly because it provided interim exposure to the public between their major books. Of course, everybody got royalties and the established authors got a bigger share.

The twelve fledgling authors were charged with driving traffic, awareness, and sales of the book their work appeared in. Meanwhile, RCS Libri worked with the powerful national newspaper in their corporate family, Corriere della Sera, to promote the You Crime series generically and run its web site.

As it turned out, all four books in the You Crime series sold quite well. They all made the top 50 (among over 4,000 titles) for Rizzoli throughout the entire Italian ebook market (including in the Kindle store). RCS Libri promoted the series as a competition, like X-Factor. The fledgling authors were expected to add their title-promotion efforts to the series branding done by Rizzoli and Corriere della Sera. And now at least some of those writers will have their own full-length novels published by Rizzoli, having been introduced to the reading public through this vehicle.

Vena calls this new form of publishing “co-publishing”, where an established publisher effectively partners with aspiring writers, bringing established writers into the project to help with their content and their brands. He sees the authors and publisher as “co-responsible” for driving readers to the book.

I don’t know whether the competitive X-Factor aspect of this or the “co-publishing” label are the key elements. Of course, they might be. But, regardless of that, the concept of using established writers to entice sampling of new writers is definitely a very cool idea, and doing this in a “digital-first” publishing paradigm, seriously reducing investment risk, makes complete sense.

Obviously, we want to see this work again before we leap to the conclusion that it will work every time, or even regularly, but having four successes out of four and a large number of fledgling writers picked up for full-novel treatment is a powerful statement on behalf of an imaginative experiment. I think we should expect to see this tried again in other markets. And before too long.

Of course, we are putting together a panel including RCS Libri at Digital Book World 2014 to talk about “no inventory publishing”. That’s one of several pieces of programming we will have around “new models”. We’ll also feature leading innovative publishers and suppliers talking about subscriptions, new direct sales channels, agile content publishing models, and new product forms for non-narrative content. Register by November 8 for the best rate. 

The three big shifts taking place in trade book publishing are very much interrelated. The fact that consumers are buying about half their books online is one. That means that publishers are not entirely dependent on books being placed at retail to make sales, which is the second. And the marketing that used to take place around store inventory is becoming digital, which is the third.

And that trinity of changes is leading to another trinity of changes inside the publishing houses as publishers find that the old rules don’t apply around what used to be three pretty-much-constants: time, timing, and budgeting.

“Time” means “time on the clock” or staff time: the hours of staff effort that marketing management has to work with. It used to be that the creative outputs required of marketers and publicists were reasonably predictable and static: jacket, title information, and press release copy and thoroughly-planned campaigns for certain books around their publication date. The marketers weren’t intended to engage intensively in interaction with media or consumers. Now, Facebook and Twitter and LinkedIn campaigns, whether they are attempting to connect through earned or paid connections, require interaction and monitoring. Staff time can be absorbed in specific campaigns. This is a management (and accounting) challenge publishers didn’t face before.

“Timing” means “time on the calendar”, or the publishers’ decisions about when it makes sense to put forth marketing effort on behalf of a book. Until very recently, publishers were justified (and often comforted) in their belief that no overt or costly marketing efforts made any sense unless inventory was on sale in bookstores. Part of the reason for the publication date-intensive marketing approach publishers have always taken is that with each day that passes after pub date, publishers have less confidence that books are available at retail locations. They reckoned, based on firm experience, that books not ubiquitously available wouldn’t sell much no matter what efforts were made to acquaint the public with the book. But with half the sales being made online (and a far higher percentage of the potential customers certainly willing to buy some of their books that way), availability in stores is no longer a sine qua non for promoting. That means that opportunities that would not have required or benefited from promotional effort in years past will now.

And that leads to the new challenges of “budgeting” for marketing efforts. Historically, publishers created a marketing budget for those books lucky enough to have one in advance of publication and spent just about all of it within weeks of the book’s release. Significant efforts just didn’t take place on any other books. That is also no longer an effective way to operate. Today’s marketers need funds (and time) available for online marketing campaigns, both coop within online retailers and more ambitious web and social network efforts, to take advantage of unpredicted newsbreaks or trends and to do seasonal or occasion-based promotions that would have, in years passed, required long planning and lead times to get inventory in place but today can be much more spontaneous.

It is also worth noting that the longstanding concepts of timing and budgeting can also be thrown into a cocked hat, but in a good way. Some digital marketing efforts will simply produce a positive ROI: the more you do it, the more money you make. Nothing lasts forever, but that can be true in a very open-ended way. When a publisher finds something like that, would their current procedures around timing and budgeting tell them to turn it off, even while it is working and producing positive margin? I’m afraid in many houses, that’s exactly what would happen.

This creates a lot of complications from the perspective of the big houses driven, as they are and must be, by signing up and then succeeding with the Big Books. The reality, which might be seen as a “dirty little secret”, has been that big books have driven their marketing efforts. Big books have provided the leverage with the stores, of course. The big house sales reps are the deliverers and deal-makers for the books which have historically driven the traffic into the stores, and that has given them the attention and leverage to get distribution across their houses’ list. Indeed, the major book review media were also responsive to the same high-profile books, which gave a house’s publicity department the same kind of bandwidth to push lower-profile titles.

If money were going to be spent, on coop for placement in stores or for ads in media, it was likely to be for the big books as well. So it was actually true that a house having major titles in its catalog used the access those titles created to generate visibility and distribution for many other titles the house published.

And the economics of big book marketing were also different, although it is not clear how much this has been taken into account. Because books that publishers and agents know will be big in advance tend to have advances calculated to be too high to earn out, a publisher can figure that all the sales margin on those titles creates a margin contribution to the publisher; the royalty is, in effect, already paid. But that’s not true further down the list, particularly on backlist, where each incremental sale can trigger an incremental royalty payment. That can confuse an ROI calculation and would tend to discourage a publisher from freely allocating money to promote backlist.

Besides the challenges around time, timing, and budgeting, publishers have a lot to change in the way they do their marketing. We’ve advocated for years that publishers look at marketing as an investment, building assets, such as email lists of consumers they can reach for free, through their marketing efforts for subsequent use. That works better if the house’s marketing efforts are vertical, or audience-centric, which enables repeat efforts to the same people to bear fruit.

Publishers are going to need new skills and new tools to be effective marketers but, even more fundamental, they have to break the mold that has placed the lion’s share of marketing effort and spend behind a small number of books in just a few short weeks around publication date.

Long before publishers become so marketing-driven that the marketers become leaders in helping to pick the books to be acquired, which we expect will happen eventually, there will need to be a complete re-think around marketing: how it is staffed, how and when campaigns should be triggered and stopped, and how both the house and the marketers themselves manage the spending to push books. Almost every marketing expert coming into big publishing from outside sees the ratio of marketers to sales personnel as wildly out of line, with far too few marketers in relation to the number of sales people. That’s because sales used to be marketing; with every bookstore that closes, that becomes less true.

Time, timing, and budgeting for marketing. Every publisher needs to be examining and changing the way they think about all three.

A big flap has arisen this week — which I believe I would have been equally aware of had I been home in New York rather than in London — because the giant UK books-and-stationery retailer WH Smith has apparently found inappropriate ebooks being recommended through the kids books portions of the Kobo-managed ebook offering they host. This has sparked a lot of conversation about how recommendations — indeed how curation — is managed in the online environment. In this case, the discussion is about the specifics of this problem and how metadata might have been wrong, gamed, misunderstood. This has resulted in Smith’s turning off their whole web site, which contains the Kobo-offered ebooks, while the problem is “fixed”. It’s a mess that points to how far we are from solving core challenges of selling books in a virtual environment.

Online bookselling has a long way to go before it can deliver even what it intends to deliver in response to a search or to prompt a next sale. Of course, there are two additional and larger problems that come first: knowing what the right suggestion(s) would be and being able to make enough of them to match the book shopping experiences online sales must replace.

Analysis offered by Russ Grandinetti of Amazon at our Publishers Launch Frankfurt conference last week suggested that the US and UK are on the verge of transacting more than 50% of the book business online, with other markets in Europe and Asia not more than two or three years behind. (This may understate the real state of affairs; in a meeting I just had in London I was told that one of the biggest UK publishers says that 60 percent of their sales of print, ebooks, and audio are through Amazon!) Online sales of books were probably in the neighborhood of 10%, or less, for most publishers a decade ago. That shift is why retail shelf space has diminished so much, with major chains having sunk in both of the big English-speaking markets (and in smaller ones as well).

When most books were bought in physical locations, it was axiomatic that a book displayed in a store had an exponentially greater chance of selling than one that wasn’t, despite wholesale supply in the US from Ingram and Baker & Taylor that could get almost any book to almost any store in 24-48 hours. It had to be seen in the store to be bought. Competent commercial trade publishers knew there was very little point to pushing a book through marketing efforts if inventory wasn’t in place at retail, because seeing the book at the time you might buy it was a more powerful trigger for purchase than any other. Indeed, all the other stimuli (reviews, suggestions from friends, conversation at the office) tended to be acted upon only when the presence in the store was in proximity to the suggestion or recommendation. (And that’s why recommendations from clerks in the store were the most powerful recommendations of all: hence the concept of “hand-selling”.)

One problem with the change to online buying from the discovery perspective is that the funnel for each shopper keeps getting narrower. It isn’t hard for somebody in a bookstore to look at hundreds of books in a few minutes. It’s nearly impossible online. This either requires the consumer to spend more time shopping to see the same number of titles they used to see in a store, or to make a decision having seen fewer. And the concern is that the decision that gets made having seen fewer can be not to buy anything at all. (Or, particularly in the case of tablet users, to buy something other than books.)

Of course, in theory, being able to present a personally-curated batch of suggestions for each customer could be far more precisely targeted than what a store can do, and, in that case, fewer titles shown might do the same job. But we are a long way from that. And, for reasons I hope this piece will make clear, personally-curated choices would actually be far more likely to be delivered by Google than by Amazon (although they would raise a host of what would be considered big privacy concerns to a lot of customers by doing it). And that’s not a reflection on the quality of anybody’s programmers, and certainly not of their commitment to their customers.

The technology that hopes to help you “pick your next book” is referred to as a “recommendation engine”. I’ve never been on the inside of such an effort but the thinking behind them seems to center around analyzing what books you’ve bought and what you’ve searched for and, from that, figuring out what you might read next. This might be based on analysis of the content itself (e.g. Pandora recommending music of similar style and quality) and/or collaborative filtering models — leveraging user inputs (purchase history, ratings, and reviews) to make recommendations for other similar users (“people who bought x also bought y”). It all recalls for me the experience of being told when I met a great bookseller, the late Joel Turner, at the 1978 American Booksellers convention in Atlanta, that “if a customer walks up to my cash register with five books, I can always sell him a sixth”.

Of course, over time, a bookseller can fill out that knowledge with even more data as they see more and more purchases and get to know their customers, and perhaps their families. But, in fact, using books bought as a guide to recommendations is an incomplete data set. It can also be a misleading one since people buy books for people other than themselves.

Another way to look at it came from my friend, Andrew Rhomberg.  Based on his experience with start-up Jellybooks, he formulated five major book discovery paths: serendipitous, social, distributed, data-driven and incentivized.

The point is that most people get their ideas about what to read next from many sources: people they talk to, reviews, news reports, business interactions. Some people say they get book recommendations from their friends; others (like me) say they don’t often read the same things their friends or relatives read. I suspect that online communities of readers tend to work best for people who do a lot of reading in genres and not nearly as well for people who mix fiction and non-fiction, entertainment and learning. And some people gravitate to what’s popular, so bestseller lists work best for them. It is clear that getting on a bestseller list fuels a book’s sales.

And books are bought for motivations other than “to read”, so it might also be important to know that a customer’s son is having a birthday, that a customer’s cousin is getting married, that a customer is shopping for a new home or looking for a new job or starting on a new hobby or spending money on an old one.

Few, if any, of these things would be apparent to even the most diligent hand-selling bookstore personnel. Bits and pieces of it might be detectable by the super-merchant Amazon (but not likely to any other).

This is one devilishly complex problem. There are countless potential inputs to the “next book purchase” decision and they are processed by each different individual in a highly personalized way. If you think it through, it seems obvious that most recommendations to most people wouldn’t work. Which takes us back to the need to make a lot of them, which a bookstore display does much better than online pages that show 10 or 20 books at one time.

In the long run, it would seem to me that Google is the entity best-positioned to address this challenge if they can somehow combine the knowledge of what you searched for (which they know), with what you read online (which they could know if you use Chrome for your browser), and the topics and book titles that have appeared in your emails (which they could know if you use Gmail) and the things you ‘like’ and talk about online (if you use Google+). Knowing your travel plans and patterns would be helpful too.

Of course, unless you use Google Play for ebook purchase and consumption, they’d be missing the two most important bits of data — what you bought and how voraciously you read it and they still wouldn’t know your print book purchases (unless they crawl your email receipts for that as well) – which Amazon is building on without all the other information. What you’d really want to do is to correlate the book buying and consumption information from the past with the behavioral data contemporary to it. With it all combined, perhaps you could filter recommendations so that the 20 or 50 you could show on line would have the commercial power of the hundreds or thousands you could see in the same amount of time in a store.

At the moment, both Amazon and Google are trying to see a pattern through one nearsighted eye.

But is this all really part of a larger problem for publishers? Is online discovery really affecting the sales patterns for books? It would appear so. One of the global ebook sellers told me during Frankfurt that their online sales are far more concentrated than publishers’ sales tended to be, with a tiny fraction of titles (under 5%) making up a huge percentage of total sales (nearly 70%). (I am assuming here that this retailer’s data is typical; of course, it may not be.) If memory serves, at the turn of the century Barnes & Noble stores saw only about 5% of their sales coming from “bestsellers” and, I believe (relying on memory of detail, which I admit is not my most powerful mental muscle) backlist outsold new titles. Publishers really live on the midlist. We know the long tail is taking an increasing share of sales and it would appear the head is too. Those sales come out of the midlist. It is pretty hard to run a profitable publisher without a profitable midlist.

And that would suggest that the increasing concentration of sales, which is likely the result of our hobbled ability to present choices in the digital sales environment, is a problem that publishers will want to address.

A somewhat overwrought article in Wired calling ebooks an “abomination” because they “price people out of reading” provokes thinking about how much the business models for the trade book business are changing. The article’s weakness stems from its focus on the pricing decisions publishers are making in selling print and ebooks to libraries when those changes are taking place in a larger and indivisible context. The industry is finding less and less uniting what it has been for the past 70 years, since the end of World War II and the advent of paperbacks, and what it will be in a future that is already being disruptive but not necessarily clear.

This is reflected on a micro level in a discussion that arose at our Marketing Conference last week from a question asking “what is a book”? That question used to have a physical answer which described an object, not necessarily describing what content it contained. We’re getting away pretty fast from requiring a book to be printed and bound; the words of an author feel no less real or worthy to many of us coming from a screen. Screen delivery is also relieving the need for a book to have any minimum length, which printed books transacted individually require for physical (we want them thick enough to bind with a spine) and commercial (selling and tracking an individual item practically requires a minimum price) reasons.

I think the questioner in this case was also trying to pull us into a disussion of video, audio, interaction, and linking, which I resist for two reasons. One is that, so far, the preponderence of ebooks that have sold any appreciable quantities have not had any of those attributes. They’re just the same words as in the printed books made reflowable for a screen. The second is that my world is the world of book publishing. My belief is that if books were to become something heavily dependent on video and audio, they won’t be made by people who today are book publishers. They’ll be made by movie studios and animation houses and digital game creators. In that case, discussion of them belongs on some other blog.

Restricting one’s thinking to assume that the future of books encompass only digital versions of what has existed as a book for the past several hundred years doesn’t, by itself, make the future clear. The changes in business models and in the configuration of the industry provide plenty of potential variation that, from my perspective, is more useful (and more fun) to think about than trying to redefine the book itself.

One of the things that has characterized books for me is the incredible diversity of markets they reach. Trade publishing has always had remarkably low barriers to entry compared to other media. It has always been easier to publish a book and make it work on some level than to launch a newspaper or a magazine, or make a movie or a TV show or a record. It costs less and the distribution channels have always been relatively democratic and accessible to outsiders. The cash comes back slowly, and profits are often elusive, but you don’t need a fortune to publish a book.

Because books inherently require a small number of sales to make money (the breakeven point gets raised by big advances to authors, but, if the author guarantee is low, most books will recover core production costs on the sale of a few thousand copies and, in some cases, less than that), they frequently target what any other industry would consider mini-markets. A publisher that mines a niche can profit on something incredibly esoteric. For example, the chances are that Osprey, a military history publisher, has made money on books about wars you’ve never heard of. But their audience has and, because they know their audience, everybody wins.

The giant general trade publishers have built big and expensive machines that can make a book a mass sensation and put it in front of the public in a big way. Other publishers have pursued other models. HarperCollins or Simon & Schuster might pay big money to an author and build an organization that can maximize marketing impact on pub date. Other companies have specialized in a market like craft books or art books or computer books, not paying the same advances and necessarily having a different emphasis in their distribution and marketing strategies.

But what has united all the business models was the commitment to make and market a book, which meant printing inventory. The minimum investment to publish a book was much less than the minimum investment to publish a magazine or a newspaper or to make a film or a record. But there still was an investment.

And that brings us back to something that made books special for their authors: the prestige conferred by somebody (preferably somebody highly professional with a brand name like some publishers have) making a unique investment in their content. That’s an investment that’s not sold as part of a magazine, or on the back of a star’s name, but in one person’s work: the author’s. When the subject of what a book was came up at our conference, one observation from a publisher of books about public affairs was how much the speaking fees of their authors went up when a book of theirs was published. The mere fact of the book conferred credibility on the author that raised their value in the marketplace, regardless of how the book sold. (Or didn’t.)

This is something inherent to the definition of a “book”. This is, also, likely to change.

The core change in publishing economics that will ultimately change the shape of the commercial industry is that the already-low investment required to publish a book has plummeted even further. As printed books become less important, then the investment required to fund them becomes less important too. Already we have seen many authors — I’ve written about John Locke and hosted Hugh Howey on the Digital Book World stage, but there are scores of others — build a career as an author without any significant print sales. We have seen other authors with long backlists, some who had only achieved modest success for publishers, turning the opportunity for higher margins and direct audience contact into financial bonanzas in digital publishing.

Repeated demonstration of the fact that it is totally possible to achieve fame and fortune as a writer without a publisher does not escape the attention of any author. Many literary agencies, the players closest to the hopes and aspirations of narrative text book authors, have been gearing up to provide digital services, primarily at first for established authors who want to self-publish their backlists. But by doing this they also create leverage for their authors in their negotiations for bigger advances and better terms from publishers, and they stamp themselves as able to continue to serve an author who decides publishers are no longer for him or her.

That means that publishers, who would theoretically always have been interested in maximizing a book’s revenue for the author and themselves, are goaded more than ever to do so. That in turn means every aspect of the business model gets questioned. Are library ebooks cannibalizing the sales of ebooks from stores? Might they? The question has to be asked. Does the fact that ebooks don’t wear out with repeated lending, as printed books do, require some different policy to make a library pony up again for frequently-loaned book? (HarperCollins has introduced such a policy.) Should a library that uses its copy of an ebook to satisfy many readers pay more than an ebook reader who has practical (and contractual) barriers to sharing? (Random House is trying this.) While some authors are asking themselves whether publishers are essential for them anymore, which makes sense, doesn’t it also make sense for publishers to be thinking hard about how the digital revolution might change their relationship with libraries?

In fact, nobody in the value chain in between the author and the reader of a book can be complacent about their position: not the agent or publisher or library, but also, quite obviously, not the bookstore, online or physical. The printer and warehouse operator must expect a shrinking share of the book business. No-inventory publishing, by lowering the barriers to entry for a written book of narrative text nearly to zero, is assuring that an ecosystem built around the reality that book inventory was the industry’s greatest cost will change profoundly.

The assertion that ebooks are making books less affordable to most people is total hogwash. For every book not available to be lent as an ebook by a library, there are probably ten from established publishers that are half the price they were before, to the consumer and to the library. And there are countless others which would not have been published before available directly from authors, which their sales tell us are valued by many readers, that are dirt cheap, priced less than the commercial transaction system for print could even consider. And the books the author of the complaining article wrote about that come with higher prices or some sort of other licensing restrictions as ebooks, are still (at least for now) still available in print at the long-traditional prices and terms.

We’re going to see marketing departments of publishers expand and sales departments contract as book distribution patterns change. We’re going to see more and more commercially viable titles launched with a no- or little-inventory-in-place model, starting with ebooks and print-on-demand availability as a low-risk launch strategy. We’re going to see books launched as serials, growing to a length determined by audience response, not based on a pre-publication plan. We’re going to see booksellers and libraries publishing and publishers building on book audiences to sell other things. And we’re going to see more and more virtual sources of books for consumers: publishers selling direct, of course, but also did you notice that Tesco is now in the game?

We’re going to see a lot of change as players of all sizes, in all parts of the publishing value chain, adjust to the “weightlessness” of a business shedding and shifting its biggest capital requirement: inventory cost. Picking on one tactic or another by one player or another, particularly from the perspective of preserving legacy behavior, is not likely to be very illuminating or helpful. The ability to put a book into the marketplace in a way that can reach more than half its audience with no inventory investment, making it possible to sell books and rights globally and only later, if it is warranted, put a bigger bet down on the book — combined with the increasing number of entities that have knowledge that could inform content and direct contact with a real market — is going to be transformative. Everybody in the chain but the author and the reader are fighting for their lives.

Smart publishers recognize that they have to completely rethink their business models and propositions in a no-inventory publishing world. Authors and agents are doing the same thing. So are many bookstores and libraries. The players in the publishing ecosystem who don’t rethink their business practices in fundamental ways will probably be relieved of the burden of thinking about them at all before long.

Branch Rickey, the fabled baseball executive who gave us racial integration, farm systems, and a host of great teams over fifty years, used to say “luck is the residue of design”. I’d like to think he was right, because we have really been lucky with our Frankfurt show for Publishers Launch, which we present in partnership with the Frankfurt Academy.

The first little lucky break was that we booked Charlie Redmayne to speak when he was CEO of Pottermore. Then earlier this summer he moved back to HarperCollins to become their UK CEO. And now his appearance at Publishers Launch Frankfurt will be his first public address since making the switch from the biggest author online play to running the UK operations for one of the industry giants.

We’d also had the idea that there were big online communities of readers that publishers could increasingly use for marketing. GoodReads had started out with the intention of being a gathering place to discuss books, but Wattpad and Scribd did not. Wattpad was a place for writers to expose their work and get critiqued by other writers; Scribd was a YouTube for documents, a place to put and find all manner of word-and-picture content online. But over time, both grew (as did GoodReads) to become large communities of word-interested people, perfect for book promotion. And when we booked them all a few months ago, both Wattpad and Scribd were well aware of the opportunity they afforded publishers.

But good luck has intervened in all three cases. GoodReads got bought by Amazon, validating (and complicating) their position as a leading gathering place for book readers. Wattpad has done a few promotional tie-ups, but a deal they did with the innovative publisher Sourcebooks that includes a line of co-published YA books and ebooks got a lot of attention. And Scribd just last week announced a new ebook subscription service, with the opening coup of landing a large number of backlist titles from HarperCollins catching everybody’s attention.

Needless to say, all three of their leaders — Otis Chandler of GoodReads, Allen Lau of Wattpad, and Trip Adler of Scribd — will have a bit more to tell our audience than we had bargained on.

We signed up Jonathan Nowell, the CEO of Nielsen Book, to talk to us about markets in transition. Nielsen has a view through both book metadata and book sales data of how markets are behaving in many countries; we wanted Jonathan to give us some clues about where we might see what has happened in the US and UK in a non-English marketplace. In the meantime, Jonathan’s company made a little fresh news too, buying the business intelligence units from Bowker in the US.

Of course, there’s a lot more at the show next Tuesday in Frankfurt. We’ll have Ken Brooks (now SVP for Global Supply Chain at McGraw-Hill) talking about how publishers should use data. We’ll have Russ Grandinetti of Amazon speaking about their view of markets in transition. Marcus Leaver of Quarto and Rebecca Smart of Osprey, two CEOs of extremely innovative global companies that are not Big Five sized, will talk about how they use being nimble and audience-focused to succeed. Micah Bowers, the CEO of Bluefire, will talk about what a DRM-free world would really be like. And Octavio Kulesz, an Argentine publisher/researcher who studies book markets in the developing world, will give us some insight into development that is quite different from what we’ve experienced in rich countries.

And we’re delighted to be hosting a panel of German publishing players about the transition in that market, which might become the first outside the English-speaking world to show real signs of disruption. It appears that this topic hasn’t even gotten as much discussion in Germany as we think it should; we’re delighted to be hosting a conversation that should be of great local interest far from where we live.

Our Frankfurt conference runs next Tuesday from 8 to 2, ending early to allow our attendees to make other meetings on what is always a busy book fair schedule.