Loud ramblings of a Software Artisan

It all started with a sketch on paper (beware of my poor penmanship).

I was thinking how I can build a musical instrument in software. Nothing new here actually, there are even applications like VMPK that fit the bill. But this is also about learning as I have started to be interested in music software.

This is how Minuit is born. The name come from a mix of a play on word where, in French, Minuit is midnight and MIDI is midday. MIDI of course is the acronym Musical Instrument Digital Interface, which is the technology at the heart of computer aided music. Also minuit sounds a bit like minuet which is a dance of social origin.

I have several goals here:

1. learn about MIDI: The application can be controlled using MIDI.
2. learn about audio: Of course you have to output audio, so now is the best time to learn about it.
3. learn about music synthesis: for now I use existing software to do so. The first instrument was ripped off Qwertone, to produce a simple tone, and the second is just Rhodes toy piano using soundfonts.
4. learn about audio plugins: the best way to bring new instruments is to use existing plugins, and there is a large selection of them that are libre.

Of course, I will use Rust for that, and that means I will have to deal with the gaps found in the Rust ecosystem, notably by interfacing libraries that are meant to be used from C or C++.

I also had to create a custom widget for the piano input, and for this I basically rewrote in Rust a widget I found in libgtkmusic that were written in Vala. That allowed me to be refine my tutorial on subclassing Gtk widgets in rust.

To add to the learning, I decided to do everything in Builder instead of my usual Emacs + Terminal combo, and I have to say it is awesome! It is good to start using tools you are not used to (ch, ch, changes!).

In end the first working version look like this:

But I didn't stop here. After some weeks off doing other things, I resumed. I was wondering, since I have a soundfont player, if I could turn this into a generic soundfont player. So I got to this:

Then iterated on the idea, learning about banks and preset for soundfont, which is straight off MIDI, and made the UI look like this:

Non configurable instruments have a placeholder:

There is no release yet, but it is getting close to the MVP. I am still missing an icon.

My focus forward will be:

1. focus on user experience. Make this a versatile musical instrument, for which the technology is a mean not an end.
2. focus on versatility by bringing more tones. I'd like to avoid the free form plugins but rather integrated plugins into the app, however leveraging that plugin architecture whenever possible.
3. explore other ideas in the musical creativity.

The repository is hosted on GNOME gitlab.

I release libgphoto2 2.2.1 yesterday after getting some report of breakage on non-Linux platforms. Nothing else has been added.

Download

Thanks to everybody for the feedback and help.

I expect to release what is in TRUNK within a month as it improve largely the support of digital audio players and as there is a huge patch to support capture with Canon 5D and 20D.

I finally took myself from a huge slacking, and I released libgphoto2 and gphoto2 2.2.0. It was long overdue.

News from 2.1.6 incude:

• support for USB Mass Storage cameras (one API for all)
• support PTP over IP
• support MTP devices (Digital Audio Players, yes)
• support lot of new cameras
• fix several outstanding bugs

Download

The more I see things going the more I think that a new project should be founded to create a unified Digital Audio Player access library. See my previous post about it.

Reading Aaron DAP probes post to provide better support of DAP to Banshee and why it must be universal, as well as James call for help for Rhythmbox real give the facts.

So here is my idea. Let's create a library that will provide a unique API to connect copy music to a Digital Audio Player. Given the various nature of the devices, it should be modular with different backends so that the maximum device number are supported. The requirement are as follow:

• LGPL licensed. As converting to "non-free" format like MP3 or AAC can be required at one point higher in the stack (like the CD ripping software), the LGPL licence allow to have such "non-free" encoder being used by the client application. That might be a problem for existing code as some are GPL licensed.
• Modular: player backends should be dynamically loaded and developped thru an API. Let's not redo the same mistakes as the Linux kernel, that API should not change (or at least minimally) in the long term as maintaining driver for hardware that gets harder to get is becoming painful. libgphoto2 quite succeeded with that, but some camera that the less modular gphoto did support are no longer in libgphoto2 for these reasons.
• Platform independent: The library should not rely on any specific UI widget set, so that any dekstop project, GNOME, KDE or whatever can use it.
• Written in C: its API shoud be language neutral and that is at the moment the best way to achieve it. C++ is not to be excluded for specific driver, but shouldn't be part of the API.
• Code reuse: whenever possible re-use existing code. This would shorten the time to produce a functionnal version of the API. And developers of such libs should unite to that project to merge all the work here.
• No kernel module dependency should be required. Only exception is Mass Storage device, but in fact there should be no reason to expose the implementation details beside obtaining the mount point from the device... which HAL and other existing pieces would provide.

I wish I had time to devote to this project, but it is not the case.

If you want to access your digital camera, you use a program that use libgphoto2. Now, with version 2.1.99 (upcoming 2.2) we even handle the camera that are mounted as a disk. It took us time to implement that because we didn't see the interest until we realized that people wanted to use their camera with a digital photo application, be it f-spot, gtkam, digikam, gthumb, and that the developer of said application shouldn't have to figure out the details. So now, we provide a single API for all the cameras that we know how to communicate with.

But what if you want to access your digital audio player (iPod, iRiver, Nomad Jukebox, etc)?

Some devices work as USB Mass Storage, like the small flash based players, hence do not require anything special driver.

For the iPods, there is libgpod that developers use to access the iPod private and proprietary database, there is ipod-sharp and its companion libipoddevice for those who write in C#, ipodslave of KDE developers (it is a kio-slave module) and gnupod, the Perl version.

For the Nomad Jukebox, there is libnjb, that implements the DPE protocol.

For the new iRiver and other MTP devices, there is experimental support in.... libgphoto2. Yes, you read correctly, a digital camera driver can be used. The MTP protocol is a variation of PTP (Picture Transfer Protocol, used for still image cameras), mostly pushed by Microsoft who provide a standard driver in the latest Windows XP updates. It is some sort of corruption of PTP and I think that the idea behind is to be able to control what files can put on the device, like implementing DRM. A bit like Apple does with iPod preventing iTunes users to download from the iPod the music, requiring that the music be on both the iPod and the computer, and having implemented a hackish crippling of iTunes to prevent loading some third-party plugins that allowed users to download the music off the iPod.

And I probably miss a lot.

How confusing is that ? Confusing for the developer that has to write at least four different version of code to access most of the device, and to the user that may not be able to use the application XyZ with the ACME player.

The solution it to create a library for digital audio player access the same way we have libgphoto2. One API for all the devices. That would involve a few changes in libgphoto2 to extract the PTP protocol, and give libgphoto2_port is own life, because the PTP implementation uses it. That would give the following architecture (admire the ASCII art):

libgphoto2 ---> ptp2camlib -> libptp2 <---- libaudioplayer
 \                   |            /           /    \
  \                  v           /           /      \
   \->      libgphoto2_port   <-/ <---------/        --> other drivers