Kindle 3.4 firmware upgrade … and using Calibre to build eBooks

If you’ve got a Kindle 3 (now called Kindle Keyboard) then there’s  a fair chance that over the past few weeks, your Kindle device took it upon itself to download a software update and a new book from Amazon to tell you how wonderful it is… it being Kindle firmware 3.4 and its support for the ‘new’ KF8 file format.

The timing could have been interesting for me. If I had bought the Kindle version of JK Rowling’s new book, Casual Vacancy, when it was first launched, then my Kindle would have displayed it perfectly. If I had bought the book two days later — following my firmware update — it would have been unreadable. You might have read about the eBook disaster that befell the Kindle and Nook launch of Casual Vacancy — the words were unreadably small (the publishers have issued a fixed version of the book, incidentally) on certain device/ firmware combinations. Such is the sometimes tricky world of eBook coding — or perhaps I should say eBook design as there are plenty of self-publishing authors who look upon eBook ‘coders’ with deep suspicion because they see no need for any coding at all, being perfectly happy to do all their work inside their word processor.

Is there really any need to get your hands dirty with html and xml coding in order to produce good eBooks, or is this a conspiracy designed to overcomplicate and obfuscate in order to justify fat fees? I’ve lost count of the number of eBooks I’ve built, but when you add them up and include all the various formats, I must be heading toward 200 books. So I feel somewhat qualified to give an opinion. I’ve a skewed view of this because I know I get the complicated books where an author has had a go, realized that what he or she wants to do isn’t as easy as they thought, and hand on to me. Certainly some of the layouts I’m asked to produce can only be built by writing code. But in many cases the work I do in html, css, or xml is the professional polish and future proofing. Most of the eBook layout work I do is inside Microsoft Word. That’s an interesting topic, but one I’ll concentrate on another day.

Today I wanted to share a thought on Calibre. This is a tool designed to help you keep track of your electronic media — newsfeeds as well as eBooks — and organise both your library and your reading devices. It’s a great tool… but for that purpose. It was never designed for publishers to build eBooks, but that’s what a lot of people use it for.

It’s tempting to use it to shortcut eBook development, something crying out to have better development toolsets. I know, I’ve been tempted myself. In retrospect I would have been better off writing my own tools to automate common coding issues such as generating the book navigation.

One route I’ve been tempted by looks like this:

  1. Do as much styling, tidying, and screening out naughty things such as tabs while still in Word or Open Office.
  2. Save out to html
  3. Import into Calibre and convert to ePUB format, taking care to turn off all the things that break Amazon Kindle coding standards, such as font scaling and setting line height.
  4. Open up the epub file in Sigil and hand code away coding problems, such as spurious blank paragraphs that have been inserted to simulate paragraph spacing for headers, fix anchor tags in the wrong place, and reinsert images. I also take a good look at the CSS and recode to match the latest Amazon guidance.
  5. Run KindleGen over the file to build the Kindle file.

In actual fact, that does work… but only for some Kindle firmware. Kindle 3.4 firmware probably does. Kindle 3.3 definitely doesn’t. At least one of the problems lies with elements with more than one class.

You what?

Here’s an example from the ePUB code of a forthcoming thriller called The Saxon Network that’s been through this route:

<p> ‘Disgraced intelligence officer wanted for murder in Italy on the kill again in World Service newsroom brawl.’ </p>

That paragraph tag has a class attribute. That’s fine… they all have that. But the ePub code gives the paragraph three classes: FirstLine, Calibre7, and SGC-2.

The problem is that Kindle firmware 3.3 only considers the first class, it always ignores subsequent classes. Kindle 3.4 will reflect all three classes. Well, that’s great, I thought; with the advent of Kindle 3.4, the Kindle devices can now understand something closer to ePUB code… I can use the Calibre-ePUB-Sigil route.

Alas — when I looked closer, some of the reader apps, such as Reader for iPad, still don’t support elements with multiple classes. The route is still not safe, at least not without careful testing.

If you’re brave and competent enough, you might want to use this route anyway. Look for examples in the code of multiple classes and recode them. Often you can spot this from the CSS. Take the class ‘Calibre7’ in the example. In the source code fed into Calibre, the paragraph was in italics, coded directly by using an tag rather than CSS. Calibre decided it would be neater to have the italics in CSS and so created a class (Calibre7) that looks like this:

 

.calibre7 {

font-style: italic

}

 

While I can see some good arguments for coding the paragraph this way, there remains a compelling reason why we mustn’t do this in the Kindle code: it simply doesn’t work, at least not for some Kindle device/ firmware combinations.

This fix in this case was simple. Locate every use of Calibre7 and replace with tags.

There will come a time when the number of Kindle devices and reader apps that do not support multiple classes will have dwindled to a small enough number that you might consider it too small a group to code for. Perhaps, but that time is not yet now.

(By the way, if you do use Calibre to build eBooks that you go on to sell, please donate to the Calibre project. Personally I pay either $2.50 or $5 per book depending on how much use I’ve made of Calibre. If I start using Sigil, I’ll do the same with them).

 

In building eBooks, I’m reminded somewhat of early software development with Microsoft Windows. My first Windows coding was for a drugs company using Windows SDK 2.1 By today’s standards it was ridiculously complicated. A ‘hello world’ program was several hundred lines of code and terribly fragile. So much of that coding environment felt unnecessarily complicated. Perhaps Windows needed to be complex, but to code Windows applications shouldn’t be. Within a few years, I was helping to roll out new development environments such as Borland Delphi, which made it possibly to write Windows applications much more quickly and deliver something more robust.

Right now we have a range of tools that don’t quite do what I want: Calibre, Sigil, Jutoh, Microsoft Word (!), InDesign. I think InDesign is probably closest, but at £450 for a licence, the price is steep for something for which the Kindle plugin is still in beta and people still report is buggy. I’m sure within five years, this situation will be a distant memory in the minds of veterans, just as it was 20 years ago with early Windows application development.

I look forward to the time when I’ll look at the newer generation of eBook designers and tell them they don’t realise how much easier they have it, just like I did with Delphi developers in the mid-90s. Until then, there is a lot to be said for those who cut away as much complexity as possible and simply hand Amazon Microsoft their Word documents to be automatically converted into Kindle books.

Advertisements

About Tim C. Taylor

Science fiction publisher and author of the bestselling Human Legion series. I live with my wife and young family in an English village. I am currently writing full time, when I'm not roped into building Lego.
This entry was posted in Writing Tips and tagged , , . Bookmark the permalink.

7 Responses to Kindle 3.4 firmware upgrade … and using Calibre to build eBooks

  1. That was a great post, Tim! Although I didn’t understand a lot of what you were discussing regarding the technical aspects, I got the gist of your thoughts. I’m so lucky that you formatted Dinosaurs & Cherry Stems for me!

  2. Laura Eno says:

    I have a Kindle 3 but I have no idea what firmware it uses. I do know that I’ve never received an update (that they’ve mentioned) or a book telling me about it.

    • timctaylor says:

      If you haven’t received a book about it yet then I guess you’re on 3.3 If you go to amazon.com, somewhere in the manage Kindle area you can choose to download the 3.4 upgrade. The 3.4, 4.1 and 5.1 look to me like they’re written using the same code stack, so if you make the upgrade and don’t see a problem with small fonts, then I think you’re safe for Kindle 4 and Kindle Touch too. You can never be entirely sure, though.
      I’m not sure there’s a way to turn off upgrades other than never having your Kindle online. They’ve probably been happening but there’s no reason for you to have noticed until the 3.4 upgrade.

      • Laura Eno says:

        I just emailed you. I never received the 3.4 because I don’t let my Kindle go to sleep! Now I have it (still no little stinkin’ book from them). The book I downloaded two weeks ago looks fine so crossing fingers.

  3. Laura Eno says:

    Huh? *facepalm* I do all of my formatting inside of MS Word. I’ve ignored KF8 completely, not wanting to be confused. I sincerely hope that this new ‘update’ won’t change the look of my books!

    • timctaylor says:

      It will look different, so I’d definitely take a look. Do you have a Kindle device? You can see how it is likely to look with Amazon Previewer running Kindle Fire device. However, the small font problem that JK Rowling had doesn’t show up in the previewer or reader app — you need to have a Kindle 3,4 or 5 running firmware 3.4, 4,1 or 5.1

      One thing that is different is the way it handles indents. In particular, if you set hanging indents for poetry, I’ve heard several people report that goes haywire.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s