Loud ramblings of a Software Artisan

Wednesday 27 April 2005

The RAW file story continued...

Nikon answer

Nikon posted some bullshit answer to justify they encrypt third-party pictures when shooting in RAW mode on D2X camera. I see people arguing that this is not the image but only the white balance. I must disagree with that, as this being part of the whole result of the shooting, it is part of the picture, hence, it is the photographer property.

Nikon's answer is simply unacceptable. Basically they consider the providing a binary SDK under NDA is a good subsitute for a fully disclosed file format, which is completely wrong. First of all, they limit your market to the one they decided for the SDK as you can't use the SDK on platform they don't support. I don't know which one as I would have to sign a NDA for that, which leads to the second point: they want to limit who can compete with that as they must approve application first. That is not free market.

As stated above, application for the Nikon SDK is possible for bona fide software companies that send Nikon a written application for the SDK. Once approved, the SDK is provided to the developer at no charge and they are authorized to use it.

What can you do ? If you are a Nikon customer, tell them that you want that documentation. This apply to other manufacturers like Canon.

BTW, if they did a Press Release, it is because they feel threatened by this information spreading, and they should be.

OpenRAW initiative

As a result, someone started the OpenRAW initiative, whose goal is to inform about the problem. This is not a new problem. Just that this time it impact a different kind of people.

Source: dpreview

Dave Coffin interview

And to finish, dpreview has an interview of Dave Coffin, dcraw author. Interesting to read.

Monday 25 April 2005

Nikon NEF "encryption" broken.

Not hot news, but News.com has an article about dcraw support of Nikon encrypted data. In short, dcraw support decrypting newer Nikon NEF (RAW) files.

RAW files thumbnailing

Despite being undocumented by the camera manufacturers, there are several Free Software tools to manipulate RAW files on Linux.

I was looking on how to provide thumbnails to RAW files in Nautilus.

I found 3 methods:

The first two method just wrap dcraw in a shell script, which is quite heavy.

The latest just take dcraw code to convert to thumbnails. It does the right thing as it extract the thumbnails from the RAW file and let GdkPixbuf convert it to the thumbnail in PNG.

I wonder why Nautilus doesn't do that for JPEG files too as most digital cameras comes with a thumbnail inside the EXIF information; falling back to the actual method if needed.

Thursday 21 April 2005

Camera manufacturers and software vendors

The gphoto project has been running without manufacturer support for at least 6 years now. No manufacturer did help us to write drivers to communicate with their camer, and only a very few have provided documentation in the past. So far, good citizens have been Kodak, who did publish the protocols for almost all their consumer camera, until they switched to PTP and reorganized themself, and Digita, that provided some sort of documentation, even if largely incomplete that was a good basis to write the driver. Other than that, it has only been either total ignorance or even some sort of hostility. One good example is Canon of such hostility, that still so far has to understand that we do their job, supporting camera they no longer provide support.

The dialog with Canon representatives, when we had one, as been useless and a waste of time. It did looked like:

- Hello I'm part of the gphoto project, we write free drivers for digital camera for Linux and other UNIX systems . We would like to support Canon cameras, so we would like to have the proper documentation for that.

- Use the Canon SDK. It is available for Windows and allow you to communicate with Canon cameras

- Mmm. As I said we run on UNIX, not Windows, so it is likely your SDK for Windows is useless to us.

- Mmm. There is a software called gphoto that works on UNIX. It might work for your. Be aware that Canon does not endorse this product at al.


- Can I have access to that SDK ?

- Which company is it for ?

- Sorry I'm an individual.

- Then I'm sorry, the SDK is not available to individuals.


Yes, they told gphoto developers that just wanted documentation to look at gphoto, and then refused us access to the SDK. But thanks to the obstination of a few smart hackers (that I was NOT part of), reverse engineering has been successfully performed to implement the Canon drivers for almost every cameras.

A couple of years after, Stephen H Westin, on behalf of Cornell university, got access to that SDK, that helped him a bit to complete the reverse engineering process towards provide software support for capturing images with a Canon camera. He admits that the documentation in the SDK, even if just documenting APIs, helped him. The documentation is still under NDA.

Nikon camera support has been somewhat easier. They started with Sierra Imaging firmware, and used this protocol, already reverse engineered by the photopc developers, then they switched to USB Mass Storage, with the advent of USB, that was already in most USB stacks, because it was part of the USB protocol suite. They never provided any assistance.

Now, with the recent Nikon story, I started collecting a bit of information. I went to Adobe user forums and found this:

From Adobe forums:

Thomas Knoll - 11:30am Mar 1, 05 PST (#3 of 18)

Adobe: Can you please document NEF format?

Nikon: No, but you can use our SDK.

Adobe: But the Nikon SDK does not provide what Camera Raw needs to operate, and using it would limit Camera Raw's speed, UI features, and quality of results (e.g. Camera > Raw's special highlight recovery algorithms).

Nikon: Then redesign Camera Raw to work within the limits of our SDK.

Adobe: No, we don't want to cripple Camera Raw. Please document NEF format.

Nikon: No, redesign Camera Raw to work within the limits of our SDK.


And I thought that with gphoto and other Free Software project, we were the only one to have problem with camera vendors to obtain the documentation and specifications for communication protocols with cameras. But Thomas Knoll, who is one of the original authors of Adobe Photoshop, seems, on behalf of Adobe, to have trouble to obtain some sort of support from Nikon to decode RAW files. Basically Nikon tell Adobe to abandon the flagship features that make Adobe software better than Nikon's (that you also have to pay extra for, shall I remind you), making it useless. Note that Free Software with dcraw did that long before Adobe release their camera RAW plugin, and of course without manufacturer support.

Can someone explain the logic behind these people to hide communication protocol, which is not the major reason to choose a product, and to not disclose file format ? I don't see any. Fortunately, in that market, there is device interroperability, like camera to printers, camera to portable drives, etc. That led to the use of the standard PTP protocol, and before that, the use of USB Mass Storage, thanks Microsoft for making it difficult to write a USB driver, and support it: manufacturers mostly switched for that reason, at least this is what I do believe, because otherwise they had no other interest given theur current attitude.

Even better. Some camera have no support for recent versions of the only OS the manufacturer support, so we end up with some request from users to port gphoto to Windows. We don't have interest in doing it ourself. One more advantage: you are not fooled by the vendor who drops support to sell upgrades.

Wednesday 20 April 2005

The Irony...

In my previous post I ranted about the fact that Nikon was making life difficult to decode RAW file from their high-end digital cameras... pointing to article telling that Adobe was mad for that. There have been a couple of insightful comments, that remind me that little ironic story.

Adobe is complaining that Nikon could use DMCA to sue them if they attempted to decrypt these data. The irony is that Adobe using that same law to sue Dmitry Sklyarov for reverse engineering the eBook file format encryption. There are affraid of the weapon they use against other people.

To not be mistaken, I'm not defending Adobe's position at all, I'm just defending photographers, aka customers, rights against an attempt from mafucturer to control the ownership of the pictures photographers take. The only product I have from Adobe is Photoshop Element 2.0 that I was illegally forced to purchase with my camera, and that I will use to see what should be done to improve free software solutions for digital photography, because lot of people claim that it is not good... and I still haven't gone past installing it.

So here is my word: Camera manufacturers, you don't own the pictures people take with your products. So you MUST provide a fully disclosed documentation of all the file formats and protocols used by your products for photographers to be able to plainly use them and exercise their rights on the photographic material they produce.

The unfair side is that Adobe did use the free software dcraw to improve their Camera RAW plugin, but dcraw author cannot use Adobe's software source code to improve his. That is one of the reason I would have released that software as GPL.

Tuesday 19 April 2005

I was always afraid of that...

...and it happens.

Thanks to Tuomas for the link.

It appears that Nikon has decided to lock down the NEF RAW file format used in their high end cameras. This is very bad. It just goes way beyond the simple dollar issues, as it seems to require people to buy the Nikon software instead of just using third parties, may it be the free dcraw or the pricey Adobe Photoshop.

It is a matter of who own the copyright on images and who own the key to view these images. With that scheme, the photographer depends on Nikon to view the images. It is like having a dependency on some magnifier manufacturer to view your printed pictures or your developed film. This has never happened. There have been some exclusive film processing like Kodachrome, but once processed, the film could be used freely like anything else.

Even worse. It just make these memories fading. Photographic archives are priceless for our history memory. And film has been somewhat appropriate to keep them, to some extent. With the era of digital picture, lot of professionnals have concerns about the perenity of the storage medias. But with Nikon initiative, that reduce it even more. In 100 years, there is absolutely no warranty that one will be able to decode these. Even in 30 years, and maybe not in 10 years.

I would have prefered the adoption of a free and documented common RAW format instead.

Photoshop news has an article.

Monday 11 April 2005

Canal Lachine, Montreal - Pictures of the day, April 11th, 2005

I went Sunday to walk along Canal Lachine in Montreal then back to downtown, and I did not have my usual camera. Here is the gallery, a little bit later than expect because I had to rotate some picture manually... unlike with my Canon 20D.

My favorite shots with the Minolta Dimage Z3:

Camera comparison

[Picture of an old camera]

I went shutterbugging today, walking along Canal Lachine in Montreal. I exchanged my EOS 20D for a Minolta Dimage Z3 and Nikon F70 (35mm film SLR), for the day. I must say I'm disappointed.

With the Nikon I found two problems. One Nikon, one Sigma:

  • The shutter trigger is too touchy. On the Canon, you half press for exposure and AF measurements. On the Nikon you just touch it. So I shot accidentally on several occasions
  • The zoom ring rotated in the wrong direction. But it was a Sigma lens.

Otherwise, it is convenient and usable. I shot 2 rolls of Black & White.

With the Minolta Dimage Z3, much more problems:

  • The view finder is electronic. You must be blind to appreciate. I just find them inconvenient to not say painfull.
  • The camera is slow: the preview freeze while autofocusing.
  • Electric zooms are slow.
  • Auto-white balance seems to be clumsy as sometime I was not getting the right tone, like pale yellow for a yellow road sign. The 20D software is much more accurate.
  • 400 ISO has a lot of noise. More than at 1600 ISO on the Canon. This size of the sensor is probably part of the reason.
  • The JPEG files don't have the picture orientation encoded. I don't know if it is the camera or that they are encoded in a proprietary way, but this is what is delaying publishing my gallery.

I shot a grand total of 257 pictures. I missed some shots because of the slow zoom and the slow auto-focus.

Sunday 3 April 2005

Cactus and Butterflies - Picture of the day, April 3rd, 2005

For once this is a really a picture of the day.

We went today to Montreal Botanical Garden were there is an exhibition with live butterflies. I did test this EF 100mm Macro f/2.8 that a friend lent me, withmy 20D. It is awesome.


The whole gallery is here.