Loud ramblings of a Software Artisan

Last week, OpenRAW.org launched their survey on the use of RAW files. Here is the press release. The idea behind that survey is to helps determine photographers practice related to digital photography, and showing what are the interest.

Calvin Jones' words about that are:

It's important to understand that we are seeking the views of all digital photographers - including those who support the OpenRAW initiative as well as those who may believe that proprietary RAW image formats are the best way for digital imaging to improve in the coming years. Our purpose is to find out what all digital photographers believe - not to hear from ONLY those who support one side of the issue. We want all voices included.

Calvin is one of the founder of the OpenRAW initiative.

Also, in the mean time, Glasslantern launched RAWPository, a repository of RAW files aimed at providing reference data sample that can be used throughout all the process writing RAW file decoders.

Ricoh GR Digital is another camera that use the documented DNG file format for RAW files. After Leica Digital module R and Hasselblad digtal backs for medium format cameras. But the Ricoh is the first to not be in the top end price range, ie much less than $1,000.

Tim Burton's latest animation movie, Corpse Bride, has been shot with a digital still camera off the shelf. The interesting part about that news is that the film makers did use the open source dcraw software for their workflow, after having used it to evaluate the cameras quality.

The irony is that this software, that none of the manufacturer support by giving the necessary documentation, made the manufacturers have their gear used for a film, with all the marketing hype they might wish (one brand won over the other).

Source: Editors Guild

Is said Nat and Larry Ewing, they plan to support RAW files.

Hey guys, do you mind joining my effort ? I currently have a request for a new project at Freedesktop.org, and I'm waiting after that to start have a common work. But if you have any other hosting solution (not sourceforhe please), you're welcome.

BTW, if you want Sigma XF3 files, there are plenty linked from there. I'm currently starting collecting them as well.

Tor, what dcoffin said about using dcraw in our code is:

• call the binary in a pipe as it outputs PPM data
• call the main() function from dcraw as he says:

Also, don't throw away my main() function -- it's full of important logic. Rename it and call it with an argv list.

• use C++ and cpp

C++ can help here. Take dcraw.c, wrap the globals in a "class DCRaw { }" declaration, add "#define CLASS DCRaw::", and you're almost done.

As far as we are concerned, none of the 3 methods allow getting metadata as the blob are extracted by parse.c, a separate program, which could get some code from the previous.

Third party programs probably use one of the last 2 methods, and resync dcraw code on a regular basis. Given the release cycle of these programs, it not a big issue. Adobe Photoshop is know to provide its own conversion methods, so they only use dcraw to decode. It is something not really well known, but something that Dave Coffin makes a living of.

When it comes to free software, UFRaw includes a now outdated version of the source code (haven't checked CVS and suspect it gets updated), and GNOME raw thumbnailer does the same with parce.c. That exhibit the real need for this library.

There is a real need for a library to provide decoding and processing, in the Free Software world. Ideally, it would just use the documentation provided by the manufacturers, but since they don't want, reverse engineering will come to the rescue. Still time to Act Now, but lest prepare plan B that can switch back to plan A.

Here is what I see is needed:

• dcraw functionnality for image processing
• meta data and MakerNote extraction from the RAW files
• output all the meta data in a portable way like XMP
• output either decoded or RAW data
• all of this in a usable C API (bindable to other language)

How can we achieve that:

• rip off dcraw code as its license allow
• use libexif (that needs to be fixed) to decode meta data and MakerNote, or any other apropriate library like the Perl ExifTool
• implement DNG encoding and decoding
• implement the missing bits like XMP

We have seen nice things in that area, and I just find that there is not enough focusing. For example Udi Fuchs, UFRaw author, wants to decode the meta data from RAW files using libexif, it is a good oportunity to put that in a library. No doubt, that will be code to reuse. UFRaw already includes the code from dcraw, why not make it a library, since Dave Coffin, from the mail exchange we have had, is not interested in at all. Apparently Udi does not want to reinvent the wheel. All of this would be a good starting point.

At GUADEC, we also had a demo of an ongoing work by Raphael Quinet, towards adding meta data support into the Gimp. If only we could share the effort....

Leica did release the Digital Module R for Leica R SLR. And as a good student, they use DNG and don't even provide their own proprietary software for them, but just a third party proprietary software: Adobe's.

Someone even posted sample files. One still have to check how good Leica developer support is towards helping development, but at least they get the start right by not using another undocumented file format. If only they could document previous RAW files they did release ?

Concerning DNG it is not an Open file format, as it is owned by Adobe, like PDF, but it is completely documented, free (of charge) to use as is, and it does not prevent any Free Software implementation from the documentation.

Jumping on the bandwagon of OpenRAW, Microsoft decide to make the PR machine work by announcing partnership with camera manufacturers to provide RAW support in the upcoming and still delayed Longhorn version of Windows. They claim that they'll solve the RAW issue by providing APIs to software vendors. Source CNet. Off course, they are proud to have attracted the major camera manufacturers that decided to be in.

That announcment is FUD. Providing a proprietary solution for the monopolistic and proprietary operating system on the market does not solve any issue that OpenRAW is trying to solve, unlike Edmund Ronal says (this article is FUD too). It just address a usability problem with RAW by being able to read them on Windows Longhorn out of the box. But Windows Longhorn does not exist.

This announcement might be even more of a bad news when it comes to free desktops like GNOME as they may get the usual answer "but we support Windows". Let's not be fooled by this. It is just part of a global strategy for world domination, and I'll do my best to help GNOME people do the same freely. Freedom is an important concern.

Before OpenRAW started their ActNow ! campaign, I requested to Canon, thru the Canon SDK e-mail address, the documentation for the RAW file format. The documentation was declined with some vague reason, and when I insisted for a clarification, because vague is not good enough, I got this:

One of the reasons we are #1 in our field is our strong portfolio of intellectual property. We devote a significant portion of our annual revenue to Research and Development, and as a result, Canon has been among the top 3 companies receiving US Patents for 13 consecutive years. The fruits of our R&D investments can be seen in many aspects of our products: We are one of the only camera manufacturers in the world who designs, develops and produces the 3 key components of our cameras, namely lenses, image sensors and image processors, as well as the software that runs them. We believe that our proprietary technologies clearly differentiate our products from those of our competitors.

In order to maintain our leadership position, we find it necessary to protect our intellectual property. Therefore, we have decided to maintain the confidential nature of the communication protocols of our digital cameras and the documentation of our RAW image data, among other things.

In our view, this has nothing to do with the ability of individuals to claim copyright to images produced by Canon cameras.

We certainly appreciate your business. However, our equipment and software is essentially offered on an "as is" basis, with no warranty as to its merchantability or fitness for a particular purpose. If our equipment or software does not meet your needs, you are entirely welcome to seek other suppliers.

Best Regards,

Chuck Westfall Director/Media & Customer Relationship Camera Marketing Group/Canon U.S.A., Inc.

(kindly suppressed e-mail and phone)

The thing is that apparently HE denied his statement later claiming it as being a personnal opinion. But this is the answer I got thru official Canon channels (it was originating from the Canon SDK, not from hime self) and that it is signed by a director, I consider it said as an official statement from Canon USA.

It is insteresting to see that now, if you ask this to the tech support (Canon Canada in my case) you get the really moderated answer:

Dear Hubert Figuiere

It is only through our customers comments and suggestions that we are able to manufacture quality products that our customers will be able to use on a consistent basis. The fact that you took the time to write to us is indeed appreciated. Please be assured that your comments have been forwarded onto the appropriate department for their information and review. Sincerely,

Which at least acknowledge that they started to recieve a lot of request from people, requiring to provide a pre-fabricated message to the support staff.

It is still time to ActNow ! if you haven't.

OpenRAW has finally started its campaign Act Now! to request the RAW file format documentation to camera manufacturers. This will be achieved with a mass effect: customer must ask it. The more will ask, the more chance this might happen.

All the details is on OpenRAW website: http://www.openraw.org/actnow/.

I did that myself, for a second time.

Engadget has an interview of Nikon folks about DSLR and the digital camera products. Engadget decided to ask the annoying question about the encrypted white balance, and Nikon PR people just say they wanted to skip the question as they have an "advisory" that answer all the questions. Apparently this matter really bug them.

BTW, Canon tech support seems to just forget to reply to customers when they find their legitimate requests annoying. More on that later.

I found some information: Foveon Inc., and their only known licensee Sigma disclosed the XF3 RAW file format with documentation and some sample images. They are just free for pickup in PDF.

I haven't read it yet, but from a company that try to innovate in the area of sensors, I find that from Foveon it is a really smart move to disclose the file format. That means they don't fear that it will be used to compete with them.

Kudos to Foveon Inc.

Today I contacted Canon Customer Support to request the documentation of the CR2 RAW file format of my Canon 20D. Here is my request:

Hi,

I would like to obtain the documentation of the Canon CR2 RAW format in order to write a software to decode and convert the pictures I produce.

Thank you.

Hub

I encourage you, as a customer, to do the same, being Canon, Nikon or whoever else. The idea is that if they get numerous requests there may be a change.

Please, once done, post a comment here.

Thanks.

Update: Here are links to Canon customer support.

• Canada
• USA
• UK
• France

Or look for Customer Support, they all have some way to ask a question.