3monkeys

Overview

In my last article, OpenOffice .odt Opened Up – Part 1: Overview, I discussed the overall package scheme for ODT documents, and pointed out that OpenOffice uses the subdocument form. In this article, I will be taking a closer look at two of simpler top level subdocuments of the four included in the specification. Specifically, we will be taking a closer look at the office:document-meta and office:document-settings elements.

As before, my test cases where produced with the following software:

• SuSE Linux 10.1
• OpenOffice 2.0.2.7.1
• zip 2.31 (March 8th 2005)

The original source document can be downloaded here oo_part1.odt, and in particular the two subdocuments under observation can be downloaded here meta.xml and settings.xml.

The office:document-meta element

The office:document-meta element provides metadata with respect to the document, such as, author, creation time and editing time, among other data. The metadata elements can be either pre-defined or user defined. Pre-defined elements should be respected and updated by the editing application. User defined elements provides a more generic way of storing and using metadata. Each user defined metadata element is compossed of a name, a type and a value. Supporting applications can access this information and display it to the user based on its type. Both pre-defined and user defined should be able to be referenced through appropriate document text fields.

The pre-defined metadata elements are largely based upon the metadata standards developed by the Dublin Core Metadata Initiative (http://www.dublincore.org), thus many of the elements use the dc namespace.

There are 18 pre-defined metadata elements, these are listed below:

• meta:generator
• dc:title
• dc:description
• dc:subject
• dc:keyword – Can appear multiple times
• meta:initial-creator
• dc:creator – Last modifier
• meta:printed-by
• meta:creation-date – Format YYYY-MM-DDThh:mm:ss
• dc:date – Last modification date, format YYYY-MM-DDThh:mm:ss
• meta:print-date
• meta:template
• meta:auto-reload
• meta-hyperlink-behaviour
• dc:language – As defined by RFC3066, with ISO 639 language code and ISO 3166 country code
• meta:editing-cycles
• meta:editing-duration – Format PnYnMnDTnHnMnS
• meta:document-statistic – Can appear multiple times, ODT attributes below
  • meta:page-count
  • meta:table-count
  • meta:draw-count
  • meta:image-count
  • meta:ole-object-count
  • meta:paragraph-count
  • meta:word-count
  • meta:character-count
  • meta:row-count
  • meta:frame-count
  • meta:sentence-count
  • meta:syllable-count
  • meta:non-whitespace-character-count

As I suggested regarding the thumbnail image in a prior article, this information could easily be extracted and displayed to users of popular search engines such as Google, Yahoo and Ask. Additionally these services could allow the user to narrow their search based on certain criteria found in the metadata.

Provided here, meta.pl, is an example written in perl using the XML::Simple package that extracts the last editor, modification date, and page and word count. If anyone would like to contribute ports of this to another language feel free. If there is significant interest, I will cover XML::Simple or other XML packages or utilities.

The office:document-settings element

Next we take a look at the office:document-settings element. This element contains application settings that may impact thedocument. It does not caontain a complete set of application settings. Being application settings, there are no particular entries that are defined ih the ODF Specification. A office:document-settings element will contain one or more config:config-item-set elements, these elements will in turn contain config:config-item, config:config-item-set, config:config-item-map-named or config:config-item-map-indexed. The discovery of how each of these elements works with a particular application, such as OpenOffice.org Writer, is left as an exersise to the reader since they do not directly affect our goals of understanding ODF as it relates to Microsoft’s .doc format. Suffice it to say, the office:document-settings element is of little interest to all but an application developer.

What’s up next?

Next up we will investigate the significantly more interesting office:document-styles element. We will also learn some optimization techniques that we can apply to this element and perhaps discover a little of how it relates to thee office:document-content element.

Until next time,

-3Monkeys

Popularity: 8% [?]

Overview

In the first article in this series, OpenOffice ODF/.odt compared to Microsoft Word .doc, I compared various file types for size efficiency. Of particular interest was the fact that OpenOffice Write stores .odts in a zip format, an implementation of PKZip to be exact. With this knowledge and the Open Document Format standard, we can investigate how certain elements of a document effect its size and overall efficiency.

My test cases where produced with the following software:

• SuSE Linux 10.1
• OpenOffice 2.0.2.7.1
• zip 2.31 (March 8th 2005)

Starting Out

As we previously observed, .odt documents are stored in ZIP format. It is possible to store the document as a single XML file that conforms to the OpenOffice.org document type definition (DTD). It is also possible to store the document as several subdocuments, each with a different document root that represents a particular aspect of the document, such as, content or style.
Quoting the Open Document Format for Office Applications (OpenDocument) v1.0 (Second Edition), (ODF Specification):

The OpenDocument format supports the following two ways of document representation:

• As a single XML document.
• As a collection of several subdocuments within a package (see section 17), each of which stores part of the complete document. Each subdocument has a different document root and stores a particular aspect of the XML document. For example, one subdocument contains the style information and another subdocument contains the content of the document. All types of documents, for example, text and spreadsheet documents, use the same document and subdocuments definitions.

There are four types of subdocuments, each with different root elements. Additionally, the single XML document has its own root element, for a total of five different supported root elements. The root elements are summarized in the following table:

Root Element	Subdocument Content	Subdoc. Name in Package
office:document	Complete office document in a single XML document.	n/a
office:document-content	Document content and automatic styles used in the content.	content.xml
office:document-styles	Styles used in the document content and automatic styles used in the styles themselves.	styles.xml
office:document-meta	Document meta information, such as the author or the time of the last save action.	meta.xml
office:document-settings	Application-specific settings, such as the window size or printer information.	settings.xml

So, what is in our reference .odt? We will use the Linux produced document from a prior article (oo_part1.odt) with XML compression disabled. We’ve done this so that the XML is more human readable. After we unzip the file using the Linux utility unzip, we have the raw files as shown below.

As you can see all four subdocuments as specified in the specification are present as well as several other files. In particular META-INF/manifest.xml list the contents of the package, including information such as full path and type.

The file Thumbnails/thumbnail.png although part of the package, is not part of the document. The thumbnail image should conform to the Thumbnail Managing Standard (TMS) at www.freedesktop.org, and therefore should be24bit, non-interlaced PNG image with full alpha transparency. The required size for the thumbnails is 128×128 pixel.

Here is the thumbnail from our reference document.

Having the thumbnail available in the package, allows other applications such as file managers to preview the document to the user. With a little creative programming, sites such as Google, Yahoo or Ask, could extract this thumbnail and preview the document for users, with little difficulty.

Document Elements

The office:document may contain any of the document elements listed below.

• office:document-attrs
• office:document-common-attrs
• office:meta
• office:settings
• office:scripts
• office:font-face-decls
• office:styles
• office:automatic-styles
• office:master-styles
• office:body

When the subdocument method is used however, elements are restricted to certain subdocuments.

Elements in content.xml

• office:document-content (subdocument root)
• office:document-common-attrs
• office:scripts
• office:font-face-decls
• office:automatic-styles
• office:body

Elements in styles.xml

• office:document-styles (subdocument root)
• office:document-attrs
• office:document-common-attrs
• office:font-face-decls
• office:styles
• office:automatic-styles
• office:master-styles

Elements in meta.xml

• office:document-meta (subdocument root)
• office:document-common-attrs
• office:meta

Elements in settings.xml

• office:document-settings (subdocument root)
• office:document-common-attrs
• office:settings

What’s Up Next?

At this point we have a clear understanding of the subdocument method that OpenOffice applies to its ODF implementation, and we know what top level elements are handled by each subdocument.

In the next article, we will ease into the subdocument elements by exploring the office:document-meta and office:document-settings elements. These two elements are rather simple and will not require as much review compared to office:document-content or office:document-styles.
Until next time.

-3Monkeys

Popularity: 23% [?]

Overview

This is the first in a series of articles that will compare ODF and in particular the OpenOffice implementation and Microsoft Office and its various data formats with respect to various measures. This article will cover the efficiency of the .odt, .doc and .xml formats, with particular interest to native and compressible file sizes.

Methodology

My windows test cases were generated using the following software:

• Microsoft Windows XP Professional 2002, SP2
• Microsoft Word 2003 (11.6368.6368) SP2
• OpenOffice 2.0.3
• Adobe Acrobat Standard 7.0.8 5/16/2006.

My Linux test cases where produced with the following software:

• SuSE Linux 10.1
• OpenOffice 2.0.2.7.1
• Adobe Reader 7.0.8 05/22/2006

I needed a fairly large chunk of text for my test, I decided on the November draft of the ISO/IEC C Standard, located at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1905.pdf (copy here). This is a significantly large document, and I decided only to use the first seven chapters for my test case. In order to produce the target documents, I selected the contents from the beginning of the document through chapter 7, and copied this to the clipboard. I then pasted the clipboard into native versions of Microsoft Word under Windows and OpenOffice Writer under both Windows and Linux. For Microsoft Word, I saved the document as a native .doc and .xml. For OpenOffice, I saved the document as native .odt and exported it as .doc. I also saved the content as .txt with Notepad under Windows as a reference point. For archival purposes, I have mirrored all documents referred to in this article on the 3monkey wiki download area.

Raw Results

File	Size
Microsoft Office .doc	921,088
Microsoft Office .xml	6,475,669
OpenOffice (XP) .odt	154,892
OpenOffice (XP) .doc	1,335,296
OpenOffice (Linux) .odt	160,045
OpenOffice (Linux) .doc	1,338,368
Notepad	417,549

Observations

My first observation was the Linux OpenOffice implementation created slightly larger file sizes than the Windows implementation. This was probably due to the differing versions. I will revisit this in a later article if it is merited.

My next observation was that the OpenOffice .doc file was significantly larger than the Microsoft Word version. This is likely due to Microsoft’s access to the complete .doc specification, and thus a better understanding of how to optimize the file content and size. For grins, I loaded the OpenOffice .doc with Microsoft Word and saved it naively. I also loaded the Microsoft Word .doc with OpenOffice and saved it both as a .doc and .odt. The results of these test are below.

File	Size
OO .doc loaded/saved in MS	808,960
MS .doc loaded/saved in OO	1,277,952
MS .doc loaded/saved as .odt in OO	155,113

This produced some interesting results. First, even though the original OpenOffice .doc file was originally larger than the native Microsoft Word version, when loaded and saved with Word, resulted in a file 12% smaller file than the original native Word .doc. This indicates that OpenOffice does not save all of the information regarding a document that Word does. This is further supported by the opposite transformation. When we load the Word document in OpenOffice and re-save as a .doc, we experience a file size reduction again. This reduction, although not as significant, clearly supports the fact that OpenOffice is not saving all the information in its .doc format as Word. By a cursory visual inspection all of the documents seem to be equivalent. Without access to the .doc file format specification it is difficult to infer whether or not the information loss is of consequence or not. In other words the file size difference may be due to bloat in the native Word format or due information loss by OpenOffice.Next most people will notice that not only is the .odt versions smaller than then .doc versions irregardless of which application produced them. Further more the .odt is almost one-third the size of the raw text from notepad. The reason the .odt is so significantly smaller is the the OpenOffice implementation applies compression on its output, and obviously decompresses it on the fly for input. This has both as advantages and disadvantages. The primary disadvantage is load and save times. Since the file must be either compressed or decompressed, this takes extra CPU cycles. However, with the speed and efficiency of today’s processors, this should be of little practical impact. The one obvious major advantage is file size. Not only does this save in raw disk storage, but also results in lower bandwidth for such mediums as email and downloads.I wondered what would the results be of compressing the .doc, .xml, .odt and .txt? I compressed all four formats using the Linux utility zip (as that is the underlining implementation for OpenOffice). The results (below) where fairly interesting and somewhat expected.

File Type	Original Size	Compressed Size
.doc	921,088	179,648
.xml	6,475,669	228,497
.odt	154,892	153,456
.txt	417,549	104,236

Notice that each format compress roughly to the same size. The .xml is larger due to both its original size thus the number of segments that needed to be compressed and additional data compared to the other formats. The .doc is roughly 15% larger than the .odt, which was only slightly compressed (perhaps to a slight algorithm change). The .txt compressed more than the others, this is due to the fact that it cares on formatting, style or meta information and is simply the raw text. Seeing the vastly decreased storage in respect to the .doc, I wonder why Microsoft does not inappropriate a compression strategy similar to OpenOffice.

Conclusion

From this limited data sample, I have to declare OpenOffice Writer the champion of round one. Perhaps if Microsoft Word employed a compressed output form the outcome may have been different. It is actually a little strange that OpenOffice which is based on a pure text format (XML) is compressed into a binary zip file and that Microsoft Word, which is a proprietary binary format is not.

What Is Up Next?

For the most part these test cases did not contain much formatting or style information, nor did it consider such elements as tables and graphs. I will investigate how these effect the efficiency in a latter article. But before I do that, I will need to expose more of how ODF works. Therefore, the next few articles in this series will be a primer for the ODF specification.

Until next time…
-3Monkeys

Popularity: 100% [?]