It's October 2023.

Niepce

Realy very little.

Fixed a cosmetic issues with icons in the workspace. The short version is that for symbolic icons to work, they must be in a "standard" path even if they are loaded from resources.

Also some minor dependency checks, etc.

lrcat-extractor

lrcat-extractor is my Rust crate to export Lightroom™ catalogs. It's used in Niepce. Just a few bits of maintenance, like replacing docopt with clap for the dumper tool, and using thiserror.

aplib-extractor

aplib-extractor is the sibling of lrcat-extractor, a crate to export Aperture™ libraries. This is actually the oldest and one of my first projects in Rust, which really shows. I worked on adding missing features so that I can implement the importer in Niepce, including listing the volumes.

The reasoning in implementing the import is to help refine the data model in Niepce to be able to handle the structure of an Aperture™ library as it offer organizational features that Ligthroom™ doesn't. This actually might lead to deeper changes in Niepce internal design.

Once the importer works, I will release the crate officially as I'll know the API is sound.

Dia on Gtk3

Two years after I started it, and mostly drop the ball, the GTK 3 port of Dia has been merged.

I also fixed a few other small issues, including some memory leaks.

If you are running Dia from the nigthly GNOME flatpak, the last update brought a big change: it is now a GTK 3 app. The UI shouldn't be much different, just some cosmetic adjustments due to theming and other like widget spacing. Also it now work with Wayland and HiDPI.

It is something I started over two years ago, and that for some reason I left on the back burner. So I did a final push and many thanks to Zander for reviewing and merging it. Now we shall hunt for regression. I already have a fix for the touchpad scrolling.

The version is numbered starting 0.98. So if it is Dia 0.98 or higher, then it's GTK 3.

There is still some more stuff to do. Fix a regression in the navigator widget, build without deprecation post GTK 3.8, etc.

Didn't really do anything on Niepce in August, so no updates on that front. However I worked on a few other things.

libopenraw

A lot of work done on it. See my other wlog post. The short version is that I should look towards releasing 0.4.0 sometime soon.

Since that last post, I have added a long standing to-do item: code to generate test cases so I can run a non regression test suite on existing files. However I get hit with flaws in the various XML serde crates.

AbiWord

Did some work on AbiWord. Nothing ground breaking, but I have a goal to try to release the next 3.0.x with one or two bug fixes (UI), and then what I call 3.next (likely 3.2) that will focus on internal changes (stability) and UI fixes (anything back-portable to 3.0.x will get back-ported).

Done

• Some code cleanup.
• Removed the use of boost in many places where the standard C++ library would work.
• Use C++17 more.
• Making the build system less recursive. It's less complex and should be more parallel.
• Removed some Gtk3 construct that are deprecated in provision of a future Gtk4 port.
• More removal of the legacy containers.
• Rewrote the toolbars to use plain buttons and a GtkBox instead of GtkToolItem. The look is slightly different.
• Fixing some Gtk3 bug: the table creation widget was not behaving properly. Will be on stable too (3.0.x)
• Fixing some bugs with styling. (also on 3.0.x)

As I previously talked about, I started porting libopenraw to Rust. It is now in a state where it has more feature than the original.

When I started writing this post, I didn't have 100% of the code Rust, but since I have removed the last bit of C++, for which I had cut corners to make sure to have a functional API for C.

The only C++ code left is the various utilities and the C++ test suite to validate.

The goal

The goal of the Rust rewrite is to have the digital camera raw parsing library written in Rust instead of C++, while still being available with a C API.

The goal of the updated C API is to be close to the original API. However it's also a good time to do some breaking changes. Notably most of the "objects" are now immutable.

Rewrite

I did the rewrite in a branch, aiming to provide the same level of functionality as the C++ code. One of the first step was to rewrite the test suite to use the same data set. Doing so did allow verifying the consistency in behaviour, a progressively implement all the features and formats.

The C++ code use inheritance a bit to override some of the behaviours based on which file format it was. One of the thing most camera raw file have in common is that they are mostly a TIFF file. However Rust doesn't do object-oriented. It has traits that allow implementing some level of polymorphism, but in that case the concept had to be rethought a little bit.

Benefits

This is where Rust shines.

First there are a lot of things Rust, or the ecosystem around, did for me. In C++ I had re-implemented a stream system to allow IO on buffers and handling endianess. In Rust you can just read from a slice (a fixed size array of data) with the Read trait, and there are built-in functions to read endian-specific values. I still have the memory of trying to figure out which include to add depending on the UNIX flavour, between Linux, macOS and the BSD.

Second, the Debug trait. Just derive Debug (it's a simple macro) and you can "print" an type like that, using string formats. In C++, well, it's a lot of work. And on enum types it will print a human readable value, ie the one you write in code.

Third, the safety. A lot of the safety feature of Rust prevent mistake. And at runtime, the bounds checking catch of a lot of things too. Being required to use Option<> or Result<> to handle cases where in C++ you'd get some null value.

At that point, a few bugs I identified porting / rewriting the code made their way to the 0.3.x series in C++.

Drawbacks

libopenraw requires a Rust compiler (technically it already did, but I know some packagers that did disable Rust. This mean that non current architecture might not be supported. To me it's not a big issue, I target modern computers.

Testing

As mentioned I rewrote the test suite with the same data set. This is an essential part of making sure the features work as previously.

I also rewrote ordiag and the dumper ordumper. The dumper allow analyzing the structure of the file, and I didn't have this one in C++ (instead I had a more primitive exifdump). The dumper is heavily inspired from exifprobe that has served me well and that I forked. Really I can see the amount of work I didn't need to do with Rust.

To add to this I wrote libopenraw-viewer, a small Rust application to view the content of camera raw files. This allow much more easily to see the output. This has helped me to find fundamental bugs in some of the parsing that led to some fixes "upstream", namely into the C++ version (0.3.x branch). I should have done that a long time ago. This also allow me to test the Rust API.

C API adaptation

Last but not least I had to provide a C API. This allow using the library.

Rust to C ffi has limitations. You can basically pass numbers, pointers, and eventually simple structures.

The libopenraw API did return several types of "ref" which are just pointers to opaque types. The idea was always to only have opaque types through the API, which internally were C++ instances.

One of the key change is that the only object that can be explicitly be created with the API is ORRawFileRef, because it's the entry point. Very few need to be released, most are held by the containing objects.

Some other constraints of the C API directed some choices in the Rust API.

• Raw files are no longer Box<> but Rc<> due to the need to retain them for the iterator.
• Metadata::String() contain a Vec<u8> instead of a String to allow for the NUL terminated strings in the C API. They are maybe not NUL terminated but ASCII.

New features

Meanwhile I added a few new features:

• extracting the white balance on most files (saveral Nikon are not yet handled).
• color converting from the camera space to sRGB on rendering.
• a multi stage processing pipeline.

This is still not enough to have a complete processing pipeline, but it's a start. It's going towards the only two issue left in the issue tracker. Not that I don't expect more but it's a nice goal post.

Integration

I submitted a PR for Miniaturo to use the Rust version of the library. This is not ready to be merged, but this actually allowed me to fix a few API bits. The new Rust API is relatively close to the old API that was the Rust to C bindings.

This is the July 2023 update for Niepce.

The importer

Where we left, the workspace tree view didn't display the hierarchy.

First, I discovered bug in the SQL triggers for folder path update. I was missing the AFTER keyword to have it run after the update.

Workspace

Second, I need to specify a parent when adding items to the model, then locate the parent, add to it. The problem is that we get a list of folders in some order (I can order by any of the SQL columns). The problem is adding an item to a parent that is not yet int the model. There are few ways to go about it.

1. Sort the folder list in order of a flattened tree. To be fair my SQL skills are not good enough, and I'm not even sure I can do that in sqlite. This would solve most of the problems.
2. Sort the folder list once received. But this might not always work.
3. Handle the case in the model: add with a placeholder parent and re-parent as needed when the parent is added.

For now I chose 3. The risk is that if the parent doesn't exist then the tree will stay lingering. It's just the view though.

Here is the result:

This is the June 2023 update for Niepce.

The importer

Back where we left, still working on the import.

It looks like there is more work than anticipated to be able to import a tree a files.

If you are looking for a Software Engineer with strong skills in C, C++, Rust and JavaScript, with a strong FLOSS background, I am available to hire.

My résumé

This is the May 2023 update for Niepce.

Life comes at you fast. And hit hard. tl;dr not as much progress as I wished: I had to put this project a bit more on the sideline.

This will be a short update.

The importer

Some small bits:

Some fixing in the metadata processing on import, notably a better handling of raw files. Turns out the previous logic broke getting metadata from video files.

Also fixing some rexiv2 / gexiv2-sys bugs that are mostly memory leaks.

I am pondering directly binding Exiv2 now that 0.28 got rid of auto_ptr<> that had been deprecated for over a decade. cxx should make this easier. This would automatically resolve the problem above, and I don't need to bind all the API, just what I need.

Forecast

To move forward the importer, I need to fix the recursive creation of folders (it currently flatten them to a single level).

Misc

There are always the other fixes.

Fedora 38

I pulled the trigger and updated to Fedora 38. And Niepce failed to build because of a bug with bindgen with clang 16, that was already fixed, triggered by libgphoto2-sys. Submitted the PR for the crate and we are good to go. The short version is that clang 16 sent different data for anonymous enums, that bindgen 0.60 couldn't handle. But at that bindgen 0.65 was fine.

This is the reason why I always check into git the generated bindings and update them as needed, instead of doing it at build time.

Application ID

I have been using org.gnome.Niepce as an application ID. While it is hosted in the main namespace repositories (it was back in the days of Subversion, to which I am thankful), Niepce is not a core app. Policies with the GNOME project do not allow using that namespace for non core apps. So I had a perform a global change rename. Not a big deal, and it doesn't change anything for users since there is no release. I just needed to get this out of the way.

libopenraw

Pushed a bit on the Rust port to implement metadata extraction. This is driven by the goal of not having three different libraries. Still some parity gaps with the C++ code, but it's closing in. I hope to be able to release 0.4.0 based on the Rust code.

Thanks you for reading.

This is the April 2023 update for Niepce. Between outages caused by an unseasonbly ice storm, squirrels chewing cable, I discovered how painful is being off the grid. This is not the excuse, just the events that lead to downtime and April not being very productive.

The importer moved a little bit forward. I tried to use the brand new "sort by date" importer tool, that is used to test and exercise the logic. The improved importer address a few long standing issues, including not using libgphoto2 for USB Mass Storage, including flash card readers. This was a shortcut I had taken and the result was suboptimal. The new approach is to use libgphoto2 to find the device, and then switch to pure filesystem operations. That was issue 26.

I picked up my camera which I haden't done much since the pandemic started. After a firmware upgrade on the Fujifilm X-T3 (it's a 4 year old model, and the firmware is from this year, unlike most smartphones), I notice a new feature that allow setting a picture as favourite. My first question was "how is it stored?". I put that on my list for later.

After taking some pictures of the newly bloomed spring flowers, I tried the importer. The new files cause a bug with metadata making the importer unable to determine the creation date ; I hit this when trying to sort these new pictures.

This is the March 2023 update for Niepce. This is not an April's fool, and this is not the year I can announce a release on April's fool day. Sorry about that.

Continuing with the renderer / previewer cache.

I had to move ncr to Rust. I didn't port it all, but the widget and the main API are now in a Rust crate npc-craw, the fourth one in the workspace. The goal of this crate is to provide the interface to the rendering pipeline. Some of the work included moving away from Cairo surface and using GdkTextures instead. The main pipeline is still the original C++ code using GEGL, it's easier for me to bind the C++ code than to write wrappers for GEGL.

In the same way, I also ported most of the darkroom module to Rust. This module is the one that will allow the image editing and currently only handle displaying the images.

All of this was necessary to finish the render / previewer integration and making it work asynchronously: the image rendering happen in the background without freezing the UI. There are still some issues but it on overall, it works well.