Loud ramblings of a Software Artisan

Sunday 27 November 2016

libopenraw 0.1.0

I just released libopenraw 0.1.0. It is to be treated as a snapshot as it hasn't reached the level of functionality I was hoping for and it has been 5 years since last release.

Head on to the download page to get a tarball.

Several new API, some API + ABI breakage. Now the .pc files are parallel installable.

Sunday 6 November 2016

Introducing GPS Ami

Once upon a time, I started geotagging my photos. For that I bought a GPS logger, an Holux M-1200E. The device works great with gpsbabel, and since my photography workflow was stuck on MacOS, I used Houdah GPS (which uses gpsbabel behind the scene, BTW). Also I have been working for too long on moving that workflow to Linux and GNOME. At one point I even started to write an app I called "Magellan" to do what that MacOS tool did, as a part of my other project, Niepce. I didn't really get motivated so it went nowhere. It was written in C++ like the rest of Niepce. The technology isn't the problem here.

Fast forward, my photography machine got upgraded to a more recent version of its operating system after the one I was using was abandonned by browser vendors, and it happens that in that upgrade, the GPS logger stopped working because MacOS 10.11 stopped providing the USB->Serial driver needed. I could install some random driver, but given how much trust I have, I decided to pass. On Linux, it still works.

I had already started rewriting Magellan in Rust using gtk-rs ; I did that as just another Rust learning project. This breakage came right as a good motivator to actually push the development of that application and make it work. And it does.

The name "Magellan" was already used for some GPS related product (not surprising), so my app became GPS Ami ("Ami" means "friend" in French).

The design

I basically reimplemented Houdah GPS, UI and such. It works OK, but I think it will act as a transitionary state. I have bigger plans.

Notably, I want to allow a better control of the device, like what bt747 can do - my logger is based on that chipset, and other automation feature so I can use GPS Ami from Niepce to download the tracks. I currently only tested with the device I have.

The implementation

As explained before GPS Ami is written in Rust and uses gtk-rs for the UI. I have to be honest, gtk-rs is not ready for prime-time, but it looks very promising and I'm very happy to be able to contribute as needed. Not surprising, but you should be ready to have to put your hands in it if you want to use it. I did just that: provided more APIs, filed some bugs, sometime fixing them. I also had to implement gudev-rs to be able to have gudev functionnality for device hotplug — to plug into the mainloop. This was a learning experience.

Rust tooling is a lot about generating a monolithic binary, without data files. This is not bad per see, but when you need data files like .ui from glade to load the UI in Gtk (albeit this not required on Gtk side, it is more convenient), you are a bit stuck. Fortunately there is the includestr!() macro in Rust which mean "load this file at compile time and put it in this string".

Another problem I had was installing the rest of the files, like the .desktop or icons, problem I solved by wrapping cargo into an automake build system.

On overall, I'm just calling gpsbabel from a UI to download file.

The future

So what should come into the future?

  • polish the UI more, possibly tweaking it. This will probably require more gtk-rs work. IMHO there are a few show stopper for a first version.
  • implement a driver for this GPS, in Rust so that the actual driver be written in memory safe language.
  • see about adding more control feature: we should be able to read the GPS values like it is done in bt747.
  • support other devices officially (I don't have any other though)

Help wanted

  • I don't have an icon
  • I only have one device to test with

Friday 7 October 2016

Rust and Automake

But why automake? Cargo is nice.

Yes it is. But it is also limited to build the Rust crate. It does one thing, very well, and easily.

Although I'm writing a GNOME application and this needs more than building the code. So I decided I need to wrap the build process into automake.

Let's start with Autoconf for Rust Project. This post is a great introduction to solving the problem and give an actual example on doing it even though the author just uses autoconf. I need automake too, but this is a good start.

We'll basically write a configure.ac and a Makefile.am in the top level Rust crate directory.

configure.ac:

AC_INIT([gpsami], m4_esyscmd([grep ^version Cargo.toml | awk '{print $3}' | tr -d '"' | tr -d "\n"]), [hub@figuiere.net])
AM_INIT_AUTOMAKE([1.11 foreign no-dependencies no-dist-gzip dist-xz subdir-objects])

Let's init autoconf and automake. We use the options: foreign to not require all the GNU files, no-dependencies because we don't have dependency tracking done by make (cargo do that for us) and subdir-objects because we have one Makefile.am and don't want recursive mode.

The m4_esyscmd macro is a shell command to extract the version out of the Cargo.toml.

VERSION=$(grep ^version Cargo.toml | awk '{print $3}' | tr -d '"' | tr -d "\n")

This does the same as above, but put it into VERSION

This shell command was adapted from Autoconf for Rust Project but fixed as it was being greedy and also captured the "version" strings from the dependencies.

AC_CHECK_PROG(CARGO, [cargo], [yes], [no])
AS_IF(test x$CARGO = xno,
    AC_MSG_ERROR([cargo is required])
)
AC_CHECK_PROG(RUSTC, [rustc], [yes], [no])
AS_IF(test x$RUSTC = xno,
    AC_MSG_ERROR([rustc is required])
)

Check for cargo and rustc. I'm pretty sure without rustc you don't have cargo, but better be safe than sorry. Note that this is considered a fatal error at configure time.

dnl Release build we do.
CARGO_TARGET_DIR=release
AC_SUBST(CARGO_TARGET_DIR)

This is a trick: we need the cargo target directory. We hardcode to release as that's what we want to build.

The end is pretty much standard.

So far just a few tricks.

Makefile.am

desktop_files = data/gpsami.desktop

desktopdir   = $(datadir)/applications
desktop_DATA = $(desktop_files)
ui_files = src/mgwindow.ui \
	$(null)

Just some basic declarations in the Makefile.am. The desktop file with installation target and the ui_files. Note that at the moment the ui files are not installed because we inline them in Rust.

EXTRA_DIST = Cargo.toml \
	src/devices.json  \
	src/devices.rs  \
	src/drivers.rs  \
	src/gpsbabel.rs  \
	src/main.rs \
	src/mgapplication.rs \
	src/utils.rs \
	$(ui_files) \
	$(desktop_in_files) \
	$(null)

We want to distribute the source files and the desktop files. This will get more complex when the crate grows as we'll need to add more files to here.

all-local:
	cargo build --release

clean-local:
	-cargo clean

Drive build and clean targets with cargo.

install-exec-local:
	$(MKDIR_P) $(DESTDIR)$(bindir)
	$(INSTALL) -c -m 755 target/@CARGO_TARGET_DIR@/gpsami $(DESTDIR)$(bindir)

We have to install the binary by hand. That's one of the drawback of cargo.

We this, we do

$ autoreconf -si
$ ./configure
$ make
# make install

This build in release and install it in the prefix. You can even make dist, which is another of the reason why I wanted to do that.

Caveats: I know this will not work if we build in a different directory than the source directory. make distcheck fails for that reason.

I'm sure there are ways to improve this, and I will probably, but I wanted to give a recipe for something I wanted to do.

Wednesday 28 September 2016

Introducing gudev-rs

A couple of weeks ago, I released gudev-rs, Rust wrappers for gudev. The goal was to be able to receive events from udev into a Gtk application written in Rust. I had a need for it, so I made it and shared it.

It is mostly auto-generated using gir-rs from the gtk-rs project. The license is MIT.

Source code

If you have problems, suggestion, patches, please feel free to submit them.

Friday 15 April 2016

Modernizing AbiWord code

When you work on a 18 year old code base like AbiWord, you encounter stuff from another age. This is the way it is in the lifecycle of software where the requirement and the tooling evolve.

Nonetheless, when AbiWord started in 1998, it was meant as a cross-platform code base written in C++ that had to compile on both Windows and Linux. C++ compiler where not as standard compliant as today so a lot of things where excluded: no template, so not standard C++ library (it was called STL at the time). Over the years, things have evolved, Mac support was added, gcc 4 got released (with much better C++ support), and in 2003 we started using template for the containers (not necessarily in that oder, BTW). Still no standard library. This came later. I just flipped the switch to make C++11 mandatory, more on that later.

As I was looking for some bugs I found it that with all that hodge podge of coding standard there wasn't any, and this caused some serious ownership problems where we'd be using freed memory. The worse is this lead to file corruption where we write garbage memory into files as are supposed to be valid XML. This is bad.

The core of the problem is the way we pass attributes / properties around. They are passed as a NULL terminated array of pointer to strings. Even index are keys, odd are string values. While keys are always considered static, values are not always. Sometime they are taken out of a std::string or a one of the custom string containers from the code base (more on that one later), sometime they are just strdup() and free() later (uh oh, memory leaks).

Maybe this is the good time to do a cleanup and modernize the code base and make sure we have safer code rather that trying to figure out one by one all the corner cases. And shall I add that there is virtually no tests on AbiWord? So it is gonna be epic.

As I'm writing this I have 8 patches with a couple very big, amounting to the following stats (from git):

134 files changed, 5673 insertions(+), 7730 deletions(-)

These numbers just show how broad the changes are, and it seems to work. The bugs I was seeing with valgrind are gone, no more access to freed memory. That's a good start.

Some of the 2000+ lines deleted are redundant code that could have been refactored (there are still a few places I marked for that), but a lot have to do with what I'm fixing. Also some changes are purely whitespace / indentation where it was relevant usually around an actual change.

Now, instead of passing around const char ** pointers, we pass around a const PP_PropertyVector & which is, currently, a typedef to std::vector<std::string>. To make things nice the main storage for these properties is now also a std::map<std::string, std::string> (possibly I will switch it to an unordered map) so that assignments are transparent to the std::string implementation. Before that it was a one of the custom containers.

Patterns like this:

 const char *props[] = { NULL, NULL, NULL, NULL };
 int i = 0;
 std::string value = object->getValue();
 props[i++] = "key";
 const char *s = strdup(value.c_str());
 props[i++] = s;
 thing->setProperties(props);
 free(s);

Turns to

 PP_PropertyValue props = {
   "key", object->getValue()
 };
 thing->setProperties(props);

Shorter, readable, less error prone. This uses C++11 initializer list. This explain some of the line removal.

Use C++ 11!

Something I can't recommend enough if you have a C++ code base is to switch to C++ 11. Amongst the new features, let me list the few that I find important:

  • auto for automatic type deduction. Make life easier in typing and also in code changes. I mostly always use it whe declaring an iterator from a container.
  • unique_ptr<> and shared_ptr<>. Smart pointer inherited from boost. But without the need for boost. unique_ptr<> replaces the dreaded auto_ptr<> that is now deprecated.
  • unordered_map<> and unordered_set<>: hash based map and set in the standard library.
  • lambda functions. Not need to explain, it was one of the big missing feature of C++ in the age of JavaScript popularity
  • move semantics: transfer the ownership of an object. Not easy to use in C++ but clearly beneficial for when you always ended up copying. This is a key part of the unique_ptr<> implementation to be usable in a container where auto_ptr<> didn't. The move semantic is the default behaviour of Rust while C++ copies.
  • initializer list allow construction of object by passing a list of initial values. I use this one a lot in this patch set for property vectors.

Don't implement your own containers.

Don't implement vector, map, set, associative container, string, lists. Use the standard C++ library instead. It is portable, it works and it likely does a better job than your own. I have another set of patches to properly remove these UT_Vector, UT_String, etc. from the AbiWord codebase. Some have been removed progressively, but it is still ongoing.

Also write tests.

This is something that is missing on AbiWord that I have tried to tackle a few time.

One more thing.

I could have mechanised these code changes to some extent, but then I wouldn't have had to review all that code in which I found issues that I addressed. Eyeball mark II is still good for that.

The patch (in progress)

Tuesday 22 March 2016

Exempi 2.3.0 and Rust...

Last week I released Exempi 2.3.0. It adds a couple more APIs and fix a few bugs.

Also I have now released my first Rust crate, that provide a Rust API to Exempi: exempi-rs. Short of rewriting the whole parsing in Rust for safety — the core of Exempi is Adobe official XMP SDK written in C++ —, this will do.

Monday 21 March 2016

Rust...

Over the last holidays I plunged and started learning Rust in a practical way. Coming from a C++ background, and having a strong dislike of the whole concept of checking the correctness at runtime, like in, say, JavaScript, Rust is really promising.

With Rust, you get:

  • raw performance since it is compiled to native code, and no garbage collection to introduce a pause.
  • RAII (Resource Acquisition Is Initialisation) that allow a clear release of resources when going out of scope, one of the major feature I like in C++.
  • Strong typing, with inference (C++ finally got the auto keyword for that purpose) the right balance between over declaration and none.
  • Ownership strictly enforced, and this is where C++ lacks: the compiler strictly enforce ownership of data, making "moveable" and "immutable" the defaults, enforcing lifetime of reference (no pointers !).
  • Relative easiness to interface foreign function (like C) with Rust, offering clear unsafe code blocks.
  • Proper tooling for dependency management, building, documentation and built-in test support.
  • A clean macro syntax, unlike the C preprocessor.
  • Concurrent programming built into a the standard library and language.

Rust is not an object oriented language. Since it doesn't have inheritance it can only do polymorphism through traits, and it has generic types. Just a design choice that force us to rethink a bit and this is checked by the compiler. And many of these design choices are here to write safer code, code less subject to causing security issues like we find everyday lately, like very recently in libotr, git and a few others.

In short, I really see myself doing more stuff with Rust.

Sunday 6 March 2016

No Flash 0.5.1

I released No Flash 0.5.1 to fix a few bugs:

  • Identify more YouTube embedding corner cases like youtube-nocookie (Issue 44)
  • Update the SDK to fix issues with recent Firefox (44 and up)

It is available on AMO now.

Friday 4 December 2015

Let's encrypt all the things

Now that letsencrypt is more widely released, I took the opportunity to generate the certificates and install them manual on my hosting. In the future I will flip the switch to force HTTPS here. For now I made sure to avoid mixed-content as much as I could.

This was long overdue.

PS: I forgot to thanks @CorySolovewicz who helped in Twitter with the problem of "invalid" private key.

Thursday 9 July 2015

No Flash 0.5 - still fighting the legacy

Last week I released No Flash 0.5, my addon for Firefox to fix the legacy of video embedding done with Flash. If you are like me and don't have Flash installed, sometime you encounter embedded video that don't work. No Flash will fix some by replacing the Flash object with a HTML5 video. This is done using the proper video embedding for HTML5.

This version brings the following:

  • Work on more recent Firefox Nightlies with e10s - it was utterly broken
  • Add support for embedded Dailymotion.

Also still, supports vimeo and YouTube - the later being extremely common.

Update: please file issues in the issue tracker.

Wednesday 13 August 2014

I was at Guadec

Guadec 2014 Volunteers

I was at Guadec in Strasbourg - thanks to all the volunteers who helped making this event possible.. For those who don't know Guadec is the annual Gnome User And Developer European Conference. I hadn't attended since 2008 — such is life — but I reconnected with people I hadn't seen in a while, as well as met awesome people that joined the project since. Attending this year made me regain a lot of motivation on why a Free Software desktop and why Gnome are really necessary. This is even more important as to why at Mozilla I use Fedora Linux rather than MacOS X like most of my team mates —*hint* at least on Fedora I don't have code signing break existing apps, and I have a real full screen browser to use to do presentation based on web technologies or even the risk that one day third party browser be barred like they are on iOS — and it is important to keep the alternatives alive. And Matthew Garrett gave us, during his keynote, good arguments on the importance of a Free Software desktop designed with users in mind.

I'll defintely try to figure out how I can make it to Göteborg, Sweden next year ; this year was facilitated by having a work week in Paris just before Guadec. Maybe I'll even present something as I resumed working on my projects.

Sunday 20 July 2014

Going to Guadec

For the first time since 2008, when it was in Istanbul, I'm coming to Guadec. This time it is in Strasbourg, France. Thanks to a work week scheduled just before in Paris.

I won't present anything this year, but I hope to be able to catch up a bit more with the Gnome community. I was already at the summit last fall, as it was being held in Montréal, but Guadec participation is usually broader and wider.

Friday 11 July 2014

Github tracks you by email.

That's right. Github tracks you by email. Each Github notification email contains in the HTML part a beacon. Beacons are usually one pixel images with a unique URL to know who did view the email or not - triggered by the HTML rendered downloading the image to display.

Two safeguards against that tracking:

  1. don't automatically download images in emails - lot of clients allow or default to this.
  2. view email only in plain text: impossible with some email system or client. Like K9-Android or just GMail. (by far this is what I do in Thunderbird)

Now I complain over twitter and according to Github Zach Holman:

"It’s a pretty rad feature for a ton of our users; reading a notification in one should mark the web UI as read too. We dig it."*.

Sorry, but there is no optout to tracking. Holman also said:

"you can just disable images. It’s the same functionality in the email as on the web, though. We’re not spying on anything."*

and

"[...] It’s just in this case there’s zero additional information trading hands."*.

Note that recent events showed me I couldn't trust Github ethics anyway, so I'd rather have them not have the info that them claiming it never change hands.

This wouldn't be important if Mozilla didn't mostly require Github to contribute to certain projects including. I filed bug 1031899. While I can understand the feature, I believe user privacy should be paramount, therefor not being able to disable tracking is a serious ethics issue.

Wednesday 4 June 2014

Hack the web: No Flash

I am a hipster Flash hater. I hated Flash before Steve Jobs told it was bad. I hate Flash before Adobe said there would be no Flash 7 for Linux. I don't have Flash on my machine. I even coined "fc;dw".

I have been muling over an idea for far too long, and an enlightning conversation with fellow Mozillians made me do it Tuesday night.

I therefor introduce a proof of concept Firefox add-on: No Flash.

The problem: there are a lot of pages around the web that embed video from big name video website, like Youtube and Vimeo. These pages might use, more often than not, the Flash embedding, either because they predate HTML5, or they use a plugin for their CMS (Wordpress) that only does that. Getting these fixed will be a pointless effort. On a positive note, it appears that Google own blogspot modified the embedding of the Youtube content they serve already.

So let's fix this on the client side.

Currently the addon will look for embedded Flash Youtube and Vimeo players and replace them in the document by the more modern iframe embedding. This has the good taste of using the vendor detection for the actual player.

Note that this is, in some way, similar to what Safari on iOS does, at least with Youtube, where they replace the Flash player with the system built-in one.

Before:

No Flash Before

Notice the Flash placeholders - I don't have Flash.

After:

No Flash After

Note the poster images.

Feel free to checkout the source code

Or file issues

Tuesday 29 April 2014

Fixing deprecations

Also, I updated the PHP version on the hosting side (the hosting company did, I just clicked on the button in the panel). This cause some glitches with the antispam and the rest when commenting. Sorry about that.

I addressed the known issues, related to deprecated PHP functions. This is still easier than upgrading to the newer version of Dotclear that break the URLs.