The route to PDF/X and where we are now: a personal history
Excerpt: Martin Bailey reflects on the history of PDF/X and its impact on production printing and the exchange of documents in PDF workflows.
About the author: Martin Bailey has worked on PDF subset standards since 1994, starting with PDF/X in CGATS and then in ISO. He’s currently the UK’s primary representative to the ISO PDF/X and … Read more
PDF is born
Way back in 1991, I remember being in the audience at Seybold San Jose. For those people younger than 50, the Seybold Report and the associated Seybold Seminars were the bible of what was happening in production printing. Jonathan Seybold was one of the first to wave a big red flag to tell printers that “the internet is coming”, and that we should all be concerned about the future of our industry. Little did he know that the internet would just change print rather than replacing it.
At that show John Warnock from Adobe took the stage and announced that they were working on what he termed “editable PostScript”. I remember feeling that he didn’t really explain in any detail what that meant. But at the time I was Technical Director for the largest PostScript service bureau in London (I suspect that might not mean too much to the under fifties, either!). I spent a portion of my time hacking PostScript produced by all the main desktop publishing tools (PageMaker, Ventura Publisher, FreeHand etc) to get them to run correctly through our RIPs. So, the idea of “editable PostScript” certainly made my ears prick up. The chief architect for the Harlequin RIP described PostScript many years later as the world’s first write-only computer programming language, because a lot of it is programmatically created and extremely hard for a human to read.
It was sometime later that Adobe finally launched PDF and what was to become Acrobat. Many in the print industry seized on it immediately as the next big step in prepress workflows. Indeed, I remember reading multiple magazine articles that said that we wouldn’t even have to preflight it and that every PDF file was guaranteed to render and print correctly. There are still times when I wish I could get a glass of whatever those writers were drinking!
In retrospect, it was obvious that the first couple of versions of PDF were simply not designed for production printing. I could never work out whether print was in the original brief but had been pushed out to a later release; or if Adobe were taken by surprise by the assumptions, complaints and demands from the print industry.
In any event, by PDF 1.2 it was getting solid enough to be used, but the workflow tools were lagging somewhat. It took until 1995 for the first PDF native RIPs to appear, most notably Jaws. Then, in 1997 Harlequin supported PDF natively, without converting it to PostScript first. In the interim it was possible to build systems to receive PDF files and to RIP them for print, but they often had a bit of a home-grown feel and sometimes home-grown reliability. It took several more years before Adobe’s own RIPs caught up.
But PDF is not enough for print
Within the first couple of years after the launch of PDF the fledgling ANSI-accredited US print standardization organization, CGATS, started a project to find a way to make it more reliable for production printing workflows. In fact, that was a follow-on to an earlier (and inconclusive) project to do the same with PostScript. Under the early leadership of Dave McDowell and Chris Goldsmith, both then at Kodak, and of Andy Masia, then at Harlequin, the committee spent several years grappling with the challenge. I joined the group sometime around 1995 and took over the chair a couple of years later.
It was challenging at least partly because none of us had experience in writing this form of technical standard for software, which was a fairly new idea across all industries back then (the ICC was formed in 1993 to define color profiles, and the W3C was founded in 1994). We were much more used to writing product requirements, and that’s a very different thing.
The whole concept of “coopetition” was also new; cooperating with our competitors to develop a common standard that would be good for all of our customers and would therefore allow us to compete on the things that matter instead of on just getting the basics right.
And thirdly, we struggled with a moving target as more and more people joined the committee and demanded that the standard address their use cases. With input from the DDAP (Digital Delivery of Advertising for Print) association under the leadership of Alan Darling we were able to confirm our initial target as display advertising for newsprint and publication (magazines).
Finally, in 1999 we published the first PDF/X (PDF for eXchange) as a CGATS standard, and a small number of vendors implemented it, both on the creation side and in consumers such as RIPs.
At that point we discovered (with apologies to Rabbie Burns) that “the best laid plans of mice and men seldom survive contact with the enemy”. We’d tried to bridge the gap between pre-existing workflows and the needs of a pure PDF workflow, and it turned out that a clean break would have been better. We’d also been led astray by poor advice on how some color management software would treat aspects of the file.
In other words, the CGATS PDF/X-1 standard could best be described as a really useful proof of concept and a learning exercise, but it was not ready for most production environments.
2000: PDF/X moves to ISO
Luckily there was interest from the international standards community, and in 2000 work started on building an improved PDF/X-1 standard in ISO, based on what we’d learned from the CGATS one. A new ISO task force was created to develop it which I chaired.
Almost immediately there was a significant political barrier. The CGATS PDF/X-1 standard was defined for delivery of documents in CMYK (optionally with spot inks), but a group of Europeans, mainly from Germany and Switzerland, felt that we should be pushing the boundaries by encouraging printers towards color managed workflows, using colors defined in Lab or with ICC profiles.
I remember sitting in a conference room in Mesa, AZ, watching the rain flooding the parking lot outside and arguing the issue backwards and forwards for hours.
In the end we defined both a CMYK PDF/X standard and a second one that allowed device independent colors. The CMYK one was closer to being ready, and was therefore published in 2001 as ISO 15930-1, defining PDF/X-1 and PDF/X-1a, and the standard allowing device-independent colors came out a year later as ISO 15930-3, defining PDF/X-3.
At the time many of us felt that publishing both was an uncomfortable compromise. But standards like PDF/X need to be pitched slightly beyond current methodologies, so that they can help to define an improved best practice. If they are not, then they provide no benefit and don’t justify implementation and use. But they must not be so far ahead that they are too far outside people’s comfort zones. The combination of PDF/X-1a for CMYK and PDF/X-3 for device-independent color was probably the best possible outcome: one that could be adopted and address an immediate need, and the second to point out the road ahead and encourage people to think seriously about it.
Of course, we were still learning, and ended up re-issuing both standards again in 2003. In particular the original PDF/X-1 conformance level was removed, leaving just PDF/X-1a in 15930-1.
2003: PDF/X proves its worth
At around that time I attended a DDAP conference in New Orleans where a speaker from a major advertising prepress house (possibly another reference that won’t mean much to younger readers?) contrasted their experience with baseline PDF, PDF/X-1a and TIFF/IT.
TIFF/IT was nominally a standardized version of TIFF, but in practice it was very focused on the delivery of display advertising and required very specialized and expensive tools to create and process it. It was popular in that field because it was virtually bulletproof. The speaker noted that their failure rate when they were sent TIFF/IT (defined as the proportion of jobs for which they had to contact the sender of the file) was less than 1%.
By contrast their failure rate with baseline PDF was somewhere around 70%. Yes, 7 in 10 files supplied as PDF needed some handholding or fixing, often because required fonts were not embedded in the file and other simple issues. No wonder companies like advertising prepress houses and print service providers were struggling to make PDF commercially viable. And this was in 2003, ten years after Adobe had launched PDF.
The good news was that the prepress house had also just run a series of tests with PDF/X-1a, and their failure rate was less than 1%; pretty much the same as TIFF/IT, but with generally cheaper and more widely available tooling, especially for viewing in approval cycles upstream before delivery.
As I remember that was pretty much the death knell for TIFF/IT, we even removed support for it from Harlequin RIPs a few years later because it just wasn’t getting used.
Flavors of PDF/X
PDF/X-1a and PDF/X-3 were both based on PDF 1.3, and the baseline PDF spec has moved on a lot since then, adding more functionality like transparency, layers etc. A PDF/X standard that didn’t allow any of those would be pretty difficult to use now. If nothing else, it would force additional approval cycles because designs using those capabilities would be transformed so much as part of exporting to PDF/X that they’d need to be signed off again.
And so additional PDF/X conformance levels have been defined over time, most obviously PDF/X-4, which enables the use of all of transparency, smooth shades, layers in a way that tries to ensure that the files will RIP as intended and expected across different products from multiple vendors.
This is always a difficult balancing act. The standard can’t afford to be left behind, but equally there should not be so many different conformance levels that people don’t know which one to use. And standards should not change under people’s feet; I remember being interviewed by a journalist about a different standard a few years ago and having to explain why updating that standard every year would be a bad thing!
There are now effectively four PDF/X standards for “blind exchange”, where everything that needs to be sent to describe the visual appearance of the graphics themselves in encapsulated into a single PDF file:
- PDF/X-1a – CMYK plus (optionally) spots, based on PDF 1.3. To be pedantic, there are actually two variants, from the 2001 and 2003 standards, but they are technically identical for all practical purposes.
- PDF/X-3 – adds support for device-independent color spaces but is also based on PDF 1.3.
- PDF/X-4 – updates the underlying PDF version to 1.6 and allows live transparency and PDF optional content (layers).
- PDF/X-6 – updates the underlying PDF version to 2.0. Provides support for color management for different substrates per page, black point compensation, metadata to describe the printed and finished piece (adding binding, folding etc to the visual appearance of the graphics themselves), etc. Also builds on the many clarifications in the underlying PDF 2.0 standard, which will, over time, ensure that products from different vendors render PDF files more consistently.
Alongside these are a handful of extra PDF/X conformance levels designed to be used in more specialist workflows, and only with the agreement of both parties in the exchange. These enable OPI-like workflows and those where many files are expected to share the same output intents. Examples include catalog work and some semi-closed publication advertising flows.
Every new conformance level needs to provide enough value to both developers and users for it to be worthwhile making the effort to implement and use it. It’s very common in such cases for the very first standard to address most of the needs of users and for every later conformance level to deliver less additional benefit, which can mean that the later ones take some time to be adopted. Personally, I don’t see that as a problem, more as an opportunity for education towards continually improving best practice, effectively continuing the route started with PDF/X-1a and PDF/X-3.
As an example, if you’re using live transparency in your designs, perhaps for something like drop shadows, you will get higher quality results if you use PDF/X-4 or PDF/X-6 instead of PDF/X-1a or PDF/X-3. That’s because the transparency won’t need to be flattened when making the PDF/X file. Flattening can give reasonable results when you know enough about the configuration of the RIP that will be used, but a key point of PDF/X is that you may not have that information. Without it your flattened transparency can show steps or jaggies.
Beyond advertising and beyond PDF/X
Pretty much everything I’ve said so far has implied a focus on classified advertising, and that was certainly the intention of the first CGATS PDF/X standard. But since then, PDF usage has spread across many areas of print, especially where the files are made in one company or department and sent to another to be printed. Even within a single organization some people use PDF/X as a useful method of enforcing self-discipline in making files that won’t fail at the RIP.
I now see PDF/X being used in direct marketing, commercial, publication, newsprint (what little there is left!), labels, packaging, wide format, product decoration and industrial print for textiles etc. In other words, PDF/X is an excellent choice in most of the print sectors where designs are delivered in PDF.
About the biggest remaining gap in PDF/X standards is around preparation of a PDF file for a specific press with extended gamut printing process inks (such as one using CMYKOGV). The PDF/X-5n conformance level goes some way towards this but is not complete. There are many who argue that it’s better for such cases to specify colors in the file using device-independent color spaces (such as wide-gamut RGB spaces or Lab), and let the prepress workflow or RIP specialize the rendering to the inks available. That makes perfect sense, especially as it makes it easier to transfer the job between presses at the last minute if required. But the PDF/X standard tends to get in the way of this approach a little bit because of the requirements around output intents. Once the color scientists agree on the data required in the PDF file and on all the necessary color transforms, maybe we’ll finally get to deliver that one, but it looks as if it’ll need a new PDF standard beyond PDF 2.0 as well.
Useful additional specifications and resources have been developed by other organizations, such as:
- the PDF/X+ specifications from the Ghent Workgroup (gwg.org), which focus on making PDF/X printable for a variety of print sectors, rather than just RIPable, which PDF/X is aimed at.
- Other groups have defined test forms that have driven improvements across the industry (most notably the Altona Test Suite).
- Additional ISO standards build on top of, or complement, PDF/X. Examples include PDF/VT (ISO 16612-2 and -3) for variable data printing, PDF Processing Steps to standardize non-printing marks, primarily for packaging (ISO 19593) and Print product metadata for PDF files (ISO 21812).
- The PDF Association is now becoming a forum for PDF experts to identify additional work required and is driving further improvements in the underlying PDF standard itself.
In other words, pretty much everyone can get involved in various aspects of PDF for production printing, and I’m proud to have been a part of the process for over a quarter of a century (gulp, is it really that long?!)