Loud ramblings of a Software Artisan

Wednesday 9 August 2017

Status update, August 2017

Work:

In March I joined the team at eyeo GmbH, the company behind Adblock Plus, as a core developer. Among other things I'm improving the filtering capabilities.

While they are based in Cologne, Germany, I'm still working remotely from Montréal.

It is great to help making the web more user centric.

Personal project:

I started working again on Niepce, currently implementing the file import. I also started to rewrite the back-end in Rust. The long term is to move completely to Rust, this will happen in parallel with feature implementation.

This and other satellite projects are part of my great plan I have for digital photography on Linux with GNOME.

'til next time.

Friday 10 March 2017

So, I got a Dell

Long. Overdue. Upgrade.

I bought a Dell XPS13 as my new portable workstation for Linux and GNOME. This is the model 9360 that is currently available as in a Developer Edition with Ubuntu 16.04 LTS (project Sputnik for those who follow). It satisifies all I was looking for in a laptop: lightweigh, small 13", 16 GB of RAM (at least), core i7 CPU (this is a Kaby Lake) and must run Linux well.

But I didn't buy the Developer Edition. Price-wise the Developer Edition is CAD$150 less than the Windows version in the "business" line and CAD$50 less than the Windows version in the "home" line (which only has that version). Exact same hardware. Last week, it was on sale, CAD$500 off the regular price, so it didn't matter. I got one. I had delayed so long for getting one, this was the opportunity for a bargain. I double checked, and unlike the previous Skylake based model that didn't have the same wifi card, this one is really the same thing.

I got a surprise door bell ring, from the delivery person (the website didn't tell me it was en route).

Unboxing

The black box

It came in a box that contain a cardboard wrap and a nice black box. The cardboard wrap contain the power brick and the AC cord. I'm genuinely surprised of the power adapter. It is small, smaller than what I'm used to. It is just odd that it doesn't come in the same box as the laptop (not a separate shipping box, just that it is boxes in a box, shipped to you at once). The nice black box only bear the Dell logo and contain the laptop, and two small booklets. One is for the warranty, the other is a getting started guide. Interestingly it mentions Ubuntu as well, which lead me to think that it is same for both preloads. This doesn't really matter in the end but it just show the level of refinement for a high-end laptop, which until the last Apple refresh, was still more expensive than the Apple equivalent.

New laptop...

Fiddling with the BIOS

It took me more time to download the Fedora live ISO and flash it to the USB stick than the actual setup of the machine, minus some fiddling. As I had booted, Fedora 25 was installed in 20 minutes. I did wipe the internal SSD, BTW.

Once I figured out it was F2 I had to press to get the BIOS upon boot, to set it to boot off the USB drive, I also had to configure the disk controller to AHCI instead of RAID, as without that the Fedora installed didn't find the internal storage. Note that this might be more troublesome if you want dual boot, but I didn't want. also I don't know what's the default setup in the Developer Edition either.

Dell XPS13 BIOS configuration

Nonetheless, the Fedora installer worked well with mostly defaults. HiDPI is managed, which that I finally have a laptop with "Retina Display".

Hands on

The laptop is small, with a HiDPI screen, a decent trackpad that works out of the box with two finger scrolling. An aluminium shell with a non glowing logo in he middle, a black inside with keyboard, and glass touch screen. The backlit keyboard has a row of function keys at the top, like it should be, row that dub as "media" button with the Fn key or actually without. Much like on a MacBook. The only difference that will require me to get used to is the Control key is actually in the corner. Like it used to be... (not sure if that's on all Dell though, but I remember hating not have control in the corner, then got used to to it like there was no option, and that was more than a decade ago). That will make for a finger reeducation, that's sure. The whole laptop a good size, it is very compact. Light but solid.

As for the connectivity, there is a SD card reader, 2 USB 3.0 port (A type) and a USB 3.1 Type-C that carries HDMI and Thunderbolt. For HDMI looks like a CAD$30 adapter, but it seems to be standard so might not be a huge problem.

Conclusion

I am happy with it. GNOME is beautiful in HiDPI, and it is all smooth.

Saturday 4 March 2017

Link: The Story of Firefox OS

Just a plain linkage to The Story of Firefox OS by Ben Francis. Ben retell the story so much better than I well ever be able to.

I’m incredibly proud of what we achieved with Firefox OS. If you measure its success against the original stated goals I think the project went way beyond its initial objectives. We went beyond a technical prototype to successfully ship 20 different commercial devices in over 30 countries, including smartphones and smart TVs. Apart from the incredible feats of engineering, prototyping 30 new APIs for the web platform and building possibly the most complex JavaScript codebase ever created, we built a mobile operating system from scratch and got it to market in less than two years. As far as I know no other team has ever done this.

I worked on Firefox OS for several years as well (I arrived on the project some time after Ben did) until mostly its end. I noticed there was a lot of misunderstanding in what the goal was, and a lot of questions. To me it was a great effort that tried to disrupt the market by opening the silos of mobile applications, using the web, trying to become the third mobile OS. A project a lot of people didn't think fit in Mozilla's mission. Its conclusion was much personal sadness.

Also remember, unlike Android, Firefox OS was developed in the true open source way: in the open, with community participation, and not behind closed doors with a code drop at each release, stripped down of features.

Saturday 17 December 2016

So, I'm looking for a job

I decided a while ago to avoid personal posts here with my loud ramblings, because they en up being pointless. I'll make an exception now and bring in a little life issue that is a little bit sudden.

I'm looking for a job, in snowy Montréal (I can do remote too).

If you are looking for an experienced developer to join your team, I can be that person. Even for a limited time, I'm ready to start as soon as you are.

My résumé

And may you have happy holidays.

Tuesday 13 December 2016

Rust and Automake - Part 2

I already talked about Rust and Automake. This is a follow-up.

One of the caveat of the previous implementation is that it doesn't pass make distcheck as it doesn't work when srcdir != builddir.

The main problem is that we need to tell cargo to build in a different directory. Fortunately Cargo PR #1657 has taken care of this. So all we need is a carefully crafted CARGO_TARGET_DIR environment variable to set before calling cargo. You can substitute abs_top_builddir for the top build directory.

But let's show the patch. Nothing complicated.

Sunday 27 November 2016

libopenraw 0.1.0

I just released libopenraw 0.1.0. It is to be treated as a snapshot as it hasn't reached the level of functionality I was hoping for and it has been 5 years since last release.

Head on to the download page to get a tarball.

Several new API, some API + ABI breakage. Now the .pc files are parallel installable.

Sunday 6 November 2016

Introducing GPS Ami

Once upon a time, I started geotagging my photos. For that I bought a GPS logger, an Holux M-1200E. The device works great with gpsbabel, and since my photography workflow was stuck on MacOS, I used Houdah GPS (which uses gpsbabel behind the scene, BTW). Also I have been working for too long on moving that workflow to Linux and GNOME. At one point I even started to write an app I called "Magellan" to do what that MacOS tool did, as a part of my other project, Niepce. I didn't really get motivated so it went nowhere. It was written in C++ like the rest of Niepce. The technology isn't the problem here.

Fast forward, my photography machine got upgraded to a more recent version of its operating system after the one I was using was abandonned by browser vendors, and it happens that in that upgrade, the GPS logger stopped working because MacOS 10.11 stopped providing the USB->Serial driver needed. I could install some random driver, but given how much trust I have, I decided to pass. On Linux, it still works.

I had already started rewriting Magellan in Rust using gtk-rs ; I did that as just another Rust learning project. This breakage came right as a good motivator to actually push the development of that application and make it work. And it does.

The name "Magellan" was already used for some GPS related product (not surprising), so my app became GPS Ami ("Ami" means "friend" in French).

The design

I basically reimplemented Houdah GPS, UI and such. It works OK, but I think it will act as a transitionary state. I have bigger plans.

Notably, I want to allow a better control of the device, like what bt747 can do - my logger is based on that chipset, and other automation feature so I can use GPS Ami from Niepce to download the tracks. I currently only tested with the device I have.

The implementation

As explained before GPS Ami is written in Rust and uses gtk-rs for the UI. I have to be honest, gtk-rs is not ready for prime-time, but it looks very promising and I'm very happy to be able to contribute as needed. Not surprising, but you should be ready to have to put your hands in it if you want to use it. I did just that: provided more APIs, filed some bugs, sometime fixing them. I also had to implement gudev-rs to be able to have gudev functionnality for device hotplug — to plug into the mainloop. This was a learning experience.

Rust tooling is a lot about generating a monolithic binary, without data files. This is not bad per see, but when you need data files like .ui from glade to load the UI in Gtk (albeit this not required on Gtk side, it is more convenient), you are a bit stuck. Fortunately there is the includestr!() macro in Rust which mean "load this file at compile time and put it in this string".

Another problem I had was installing the rest of the files, like the .desktop or icons, problem I solved by wrapping cargo into an automake build system.

On overall, I'm just calling gpsbabel from a UI to download file.

The future

So what should come into the future?

  • polish the UI more, possibly tweaking it. This will probably require more gtk-rs work. IMHO there are a few show stopper for a first version.
  • implement a driver for this GPS, in Rust so that the actual driver be written in memory safe language.
  • see about adding more control feature: we should be able to read the GPS values like it is done in bt747.
  • support other devices officially (I don't have any other though)

Help wanted

  • I don't have an icon
  • I only have one device to test with

Friday 7 October 2016

Rust and Automake

But why automake? Cargo is nice.

Yes it is. But it is also limited to build the Rust crate. It does one thing, very well, and easily.

Although I'm writing a GNOME application and this needs more than building the code. So I decided I need to wrap the build process into automake.

Let's start with Autoconf for Rust Project. This post is a great introduction to solving the problem and give an actual example on doing it even though the author just uses autoconf. I need automake too, but this is a good start.

We'll basically write a configure.ac and a Makefile.am in the top level Rust crate directory.

configure.ac:

AC_INIT([gpsami], m4_esyscmd([grep ^version Cargo.toml | awk '{print $3}' | tr -d '"' | tr -d "\n"]), [hub@figuiere.net])
AM_INIT_AUTOMAKE([1.11 foreign no-dependencies no-dist-gzip dist-xz subdir-objects])

Let's init autoconf and automake. We use the options: foreign to not require all the GNU files, no-dependencies because we don't have dependency tracking done by make (cargo do that for us) and subdir-objects because we have one Makefile.am and don't want recursive mode.

The m4_esyscmd macro is a shell command to extract the version out of the Cargo.toml.

VERSION=$(grep ^version Cargo.toml | awk '{print $3}' | tr -d '"' | tr -d "\n")

This does the same as above, but put it into VERSION

This shell command was adapted from Autoconf for Rust Project but fixed as it was being greedy and also captured the "version" strings from the dependencies.

AC_CHECK_PROG(CARGO, [cargo], [yes], [no])
AS_IF(test x$CARGO = xno,
    AC_MSG_ERROR([cargo is required])
)
AC_CHECK_PROG(RUSTC, [rustc], [yes], [no])
AS_IF(test x$RUSTC = xno,
    AC_MSG_ERROR([rustc is required])
)

Check for cargo and rustc. I'm pretty sure without rustc you don't have cargo, but better be safe than sorry. Note that this is considered a fatal error at configure time.

dnl Release build we do.
CARGO_TARGET_DIR=release
AC_SUBST(CARGO_TARGET_DIR)

This is a trick: we need the cargo target directory. We hardcode to release as that's what we want to build.

The end is pretty much standard.

So far just a few tricks.

Makefile.am

desktop_files = data/gpsami.desktop

desktopdir   = $(datadir)/applications
desktop_DATA = $(desktop_files)
ui_files = src/mgwindow.ui \
	$(null)

Just some basic declarations in the Makefile.am. The desktop file with installation target and the ui_files. Note that at the moment the ui files are not installed because we inline them in Rust.

EXTRA_DIST = Cargo.toml \
	src/devices.json  \
	src/devices.rs  \
	src/drivers.rs  \
	src/gpsbabel.rs  \
	src/main.rs \
	src/mgapplication.rs \
	src/utils.rs \
	$(ui_files) \
	$(desktop_in_files) \
	$(null)

We want to distribute the source files and the desktop files. This will get more complex when the crate grows as we'll need to add more files to here.

all-local:
	cargo build --release

clean-local:
	-cargo clean

Drive build and clean targets with cargo.

install-exec-local:
	$(MKDIR_P) $(DESTDIR)$(bindir)
	$(INSTALL) -c -m 755 target/@CARGO_TARGET_DIR@/gpsami $(DESTDIR)$(bindir)

We have to install the binary by hand. That's one of the drawback of cargo.

We this, we do

$ autoreconf -si
$ ./configure
$ make
# make install

This build in release and install it in the prefix. You can even make dist, which is another of the reason why I wanted to do that.

Caveats: I know this will not work if we build in a different directory than the source directory. make distcheck fails for that reason.

I'm sure there are ways to improve this, and I will probably, but I wanted to give a recipe for something I wanted to do.

Wednesday 28 September 2016

Introducing gudev-rs

A couple of weeks ago, I released gudev-rs, Rust wrappers for gudev. The goal was to be able to receive events from udev into a Gtk application written in Rust. I had a need for it, so I made it and shared it.

It is mostly auto-generated using gir-rs from the gtk-rs project. The license is MIT.

Source code

If you have problems, suggestion, patches, please feel free to submit them.

Friday 15 April 2016

Modernizing AbiWord code

When you work on a 18 year old code base like AbiWord, you encounter stuff from another age. This is the way it is in the lifecycle of software where the requirement and the tooling evolve.

Nonetheless, when AbiWord started in 1998, it was meant as a cross-platform code base written in C++ that had to compile on both Windows and Linux. C++ compiler where not as standard compliant as today so a lot of things where excluded: no template, so not standard C++ library (it was called STL at the time). Over the years, things have evolved, Mac support was added, gcc 4 got released (with much better C++ support), and in 2003 we started using template for the containers (not necessarily in that oder, BTW). Still no standard library. This came later. I just flipped the switch to make C++11 mandatory, more on that later.

As I was looking for some bugs I found it that with all that hodge podge of coding standard there wasn't any, and this caused some serious ownership problems where we'd be using freed memory. The worse is this lead to file corruption where we write garbage memory into files as are supposed to be valid XML. This is bad.

The core of the problem is the way we pass attributes / properties around. They are passed as a NULL terminated array of pointer to strings. Even index are keys, odd are string values. While keys are always considered static, values are not always. Sometime they are taken out of a std::string or a one of the custom string containers from the code base (more on that one later), sometime they are just strdup() and free() later (uh oh, memory leaks).

Maybe this is the good time to do a cleanup and modernize the code base and make sure we have safer code rather that trying to figure out one by one all the corner cases. And shall I add that there is virtually no tests on AbiWord? So it is gonna be epic.

As I'm writing this I have 8 patches with a couple very big, amounting to the following stats (from git):

134 files changed, 5673 insertions(+), 7730 deletions(-)

These numbers just show how broad the changes are, and it seems to work. The bugs I was seeing with valgrind are gone, no more access to freed memory. That's a good start.

Some of the 2000+ lines deleted are redundant code that could have been refactored (there are still a few places I marked for that), but a lot have to do with what I'm fixing. Also some changes are purely whitespace / indentation where it was relevant usually around an actual change.

Now, instead of passing around const char ** pointers, we pass around a const PP_PropertyVector & which is, currently, a typedef to std::vector<std::string>. To make things nice the main storage for these properties is now also a std::map<std::string, std::string> (possibly I will switch it to an unordered map) so that assignments are transparent to the std::string implementation. Before that it was a one of the custom containers.

Patterns like this:

 const char *props[] = { NULL, NULL, NULL, NULL };
 int i = 0;
 std::string value = object->getValue();
 props[i++] = "key";
 const char *s = strdup(value.c_str());
 props[i++] = s;
 thing->setProperties(props);
 free(s);

Turns to

 PP_PropertyValue props = {
   "key", object->getValue()
 };
 thing->setProperties(props);

Shorter, readable, less error prone. This uses C++11 initializer list. This explain some of the line removal.

Use C++ 11!

Something I can't recommend enough if you have a C++ code base is to switch to C++ 11. Amongst the new features, let me list the few that I find important:

  • auto for automatic type deduction. Make life easier in typing and also in code changes. I mostly always use it whe declaring an iterator from a container.
  • unique_ptr<> and shared_ptr<>. Smart pointer inherited from boost. But without the need for boost. unique_ptr<> replaces the dreaded auto_ptr<> that is now deprecated.
  • unordered_map<> and unordered_set<>: hash based map and set in the standard library.
  • lambda functions. Not need to explain, it was one of the big missing feature of C++ in the age of JavaScript popularity
  • move semantics: transfer the ownership of an object. Not easy to use in C++ but clearly beneficial for when you always ended up copying. This is a key part of the unique_ptr<> implementation to be usable in a container where auto_ptr<> didn't. The move semantic is the default behaviour of Rust while C++ copies.
  • initializer list allow construction of object by passing a list of initial values. I use this one a lot in this patch set for property vectors.

Don't implement your own containers.

Don't implement vector, map, set, associative container, string, lists. Use the standard C++ library instead. It is portable, it works and it likely does a better job than your own. I have another set of patches to properly remove these UT_Vector, UT_String, etc. from the AbiWord codebase. Some have been removed progressively, but it is still ongoing.

Also write tests.

This is something that is missing on AbiWord that I have tried to tackle a few time.

One more thing.

I could have mechanised these code changes to some extent, but then I wouldn't have had to review all that code in which I found issues that I addressed. Eyeball mark II is still good for that.

The patch (in progress)

Tuesday 22 March 2016

Exempi 2.3.0 and Rust...

Last week I released Exempi 2.3.0. It adds a couple more APIs and fix a few bugs.

Also I have now released my first Rust crate, that provide a Rust API to Exempi: exempi-rs. Short of rewriting the whole parsing in Rust for safety — the core of Exempi is Adobe official XMP SDK written in C++ —, this will do.

Monday 21 March 2016

Rust...

Over the last holidays I plunged and started learning Rust in a practical way. Coming from a C++ background, and having a strong dislike of the whole concept of checking the correctness at runtime, like in, say, JavaScript, Rust is really promising.

With Rust, you get:

  • raw performance since it is compiled to native code, and no garbage collection to introduce a pause.
  • RAII (Resource Acquisition Is Initialisation) that allow a clear release of resources when going out of scope, one of the major feature I like in C++.
  • Strong typing, with inference (C++ finally got the auto keyword for that purpose) the right balance between over declaration and none.
  • Ownership strictly enforced, and this is where C++ lacks: the compiler strictly enforce ownership of data, making "moveable" and "immutable" the defaults, enforcing lifetime of reference (no pointers !).
  • Relative easiness to interface foreign function (like C) with Rust, offering clear unsafe code blocks.
  • Proper tooling for dependency management, building, documentation and built-in test support.
  • A clean macro syntax, unlike the C preprocessor.
  • Concurrent programming built into a the standard library and language.

Rust is not an object oriented language. Since it doesn't have inheritance it can only do polymorphism through traits, and it has generic types. Just a design choice that force us to rethink a bit and this is checked by the compiler. And many of these design choices are here to write safer code, code less subject to causing security issues like we find everyday lately, like very recently in libotr, git and a few others.

In short, I really see myself doing more stuff with Rust.

Sunday 6 March 2016

No Flash 0.5.1

I released No Flash 0.5.1 to fix a few bugs:

  • Identify more YouTube embedding corner cases like youtube-nocookie (Issue 44)
  • Update the SDK to fix issues with recent Firefox (44 and up)

It is available on AMO now.

Friday 4 December 2015

Let's encrypt all the things

Now that letsencrypt is more widely released, I took the opportunity to generate the certificates and install them manual on my hosting. In the future I will flip the switch to force HTTPS here. For now I made sure to avoid mixed-content as much as I could.

This was long overdue.

PS: I forgot to thanks @CorySolovewicz who helped in Twitter with the problem of "invalid" private key.

Thursday 9 July 2015

No Flash 0.5 - still fighting the legacy

Last week I released No Flash 0.5, my addon for Firefox to fix the legacy of video embedding done with Flash. If you are like me and don't have Flash installed, sometime you encounter embedded video that don't work. No Flash will fix some by replacing the Flash object with a HTML5 video. This is done using the proper video embedding for HTML5.

This version brings the following:

  • Work on more recent Firefox Nightlies with e10s - it was utterly broken
  • Add support for embedded Dailymotion.

Also still, supports vimeo and YouTube - the later being extremely common.

Update: please file issues in the issue tracker.