Tuesday, August 19, 2014

Making an LV2 plugin GUI (yes, in Inkscape)

Told you I'd be back.

So I made this pretty UI last post but I never really told you how to actually use it (I'm assuming you've read the previous 2 posts, this is the 3rd in the series). Since I'm "only a plugin developer," that's what I'm going to apply it to. Now I've been making audio plugins for longer than I can hold my breath, but I've never bothered to make one with a GUI. GUI coding seems so boring compared to DSP and it's so subjective (user: "that GUI is so unintuitive/natural/cluttered/inefficient/pretty/ugly/slow etc. etc....") and I actually like the idea of using your ears rather than a silly visual curve, but I can't deny, a pretty GUI does increase usership. Look at the Calf plugins...

Anyhow, regardless of whether its right or wrong I'm going to make GUIs (that are completely optional, you can always use the host generated UI). I think with the infamous cellular automaton synth I will actually be able to make it easier to use, so the GUI is justifiable, but other than that they're all eye candy, so why not make 'em sweet? So I'll draw them first, then worry about making them an actual UI. I've been trying to do this drawing-first strategy for years but once I started toying with svg2cairo I thought I might actually be able to do it this time. Actually as I'm writing this paragraph the ball is still up in the air, so it might not pan out, but I'm pretty confident by the time you read the last paragraph in this almost-tutorial I'll have a plugin with a GUI.

(*EDIT 14 Sept 2015 - a big mistake was pointed out to me in my LV2_UI instantiation, updated below).

So lets rip into it:


One challenge I have is that I really don't like coding C++ much. I'm pretty much a C purist. So why didn't I use gtk? Well, cause it didn't have AVTK. Or ntk-fluid. With that fill-in-the-blank development style fluid lends to, I barely even notice that its C++ going on in back. Its a pretty quick process too. I had learned a fair bit of QT, but was forgoing that anyway, but with these new (to me) tools I had a head start and got to where I am relatively quickly (considering my qsvg widgets are now 3 years old and unfinished).

The other good news is that the DSP and UI are separate binaries and can have completely separate codebases, so I can still do the DSP in my preferred language. This forced separation is very good practice for realtime signal processing. DSP should be the top priority and should never ever ever have to wait for the GUI for anything.

But anyway, to make an LV2 plugin gui we'll need to add some extra .ttl stuff. So in manifest.ttl:
@prefix lv2:  <http://lv2plug.in/ns/lv2core#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix ui:   <http://lv2plug.in/ns/extensions/ui#> .
 
 <http://infamousplugins.sourceforge.net/plugs.html#stuck>
        a lv2:Plugin, lv2:DelayPlugin ;
        lv2:binary <stuck.so> ;
        rdfs:seeAlso <stuck.ttl> .
 
<http://infamousplugins.sourceforge.net/plugs.html#stuck_ui>
        a ui:X11UI;
        ui:binary <stuckui.so>
        lv2:extensionData ui:idle; . 

Thats not a big departure from the no-UI version, but we'd better make a stuckui.so to back it up. We've got a .cxx and .h from ntk-fluid that we made in the previous 2 posts, but its not going to be enough. The callbacks need to do something. But what? Well, they will be passing values into the control ports of the plugin DSP somehow. OpenAVproductions genius Harry Haaren wrote a little tutorial on it. The thing is called a write function. Each port has an index assigned by the .ttl and the dsp source usually has an enum to keep these numbers labeled. So include (or copy) this enum in the UI code, declare an LV2UI_Write_Function and also an LV2UI_Controller that will get passed in as an argument to the function. Both of these will get initialized with arguments that get passed in from the host when the UI instantiate function is called. The idea is the LV2_Write_Function is a function pointer that will call something from the host that stuffs data into the port. You don't need to worry about how that function works, just feel comfort knowing that where ever that points, it'll take care of you. In a thread safe way even.

Another detail (that I forgot when I first posted this yesterday) is declaring that this plugin will use the UI you define in the manifest.ttl. What that means is in the stuck.ttl you add the ui extension and declare the STUCKURI as the UI for this plugin:
@prefix doap:  <http://usefulinc.com/ns/doap#> .
@prefix foaf:  <http://xmlns.com/foaf/0.1/> .
@prefix rdf:   <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .

@prefix lv2:   <http://lv2plug.in/ns/lv2core#> .
@prefix ui:   <http://lv2plug.in/ns/extensions/ui#> .

<http://infamousplugins.sourceforge.net/plugs.html#stuck>
        a lv2:Plugin, lv2:DelayPlugin ;
        doap:name "the infamous stuck" ;
        doap:maintainer [
                foaf:name "Spencer Jackson" ;
                foaf:homepage <http://infamousplugins.sourceforge.net> ;
                foaf:mbox <ssjackson71@gmail.com> ;
        ] ;
        lv2:requiredFeature <http://lv2plug.in/ns/ext/urid#map> ;
        lv2:optionalFeature lv2:hardRTCapable ;
        ui:ui <http://infamousplugins.sourceforge.net/plugs.html#stuck_ui> ;

        lv2:port [
... 

So enough talk. Lets code.
For LV2 stuff we need an additional header. So in an extra code box (I used the window's):
#include "lv2/lv2plug.in/ns/extensions/ui/ui.h"


It will be convenient to share a single source file for which port is which index. That eliminates room for error if anything changes. So in an additional code box (the Aspect Group's since the window's are all full):
#include"stuck.h"
 
We also will need 2 additional members in our StuckUI class. Do this by adding 2 "declarations" in fltk. The code is:
LV2UI_Write_Function write_function;

and
LV2UI_Controller controller;

And finally in each callback add something along the lines of (i.e. for the Stick It! port):
write_function(controller,STICKIT,sizeof(float),0,&stickit->floatvalue);

This is calling the write function with the controller object, port number, "buffer" size (usually the size of float), protocol (usually 0, for float), and pointer to a "buffer" as arguments. So now when the button is clicked it will pass the new value on to the DSP in a threadsafe way. The official documentation of write functions is here. The floatvalue member of dials and buttons is part of ffffltk (which was introduced in the other parts of this series) which was added exclusively for LV2 plugins. Cause they always work in floats. Or in atoms, which is a whole other ball of wax. Really though, its really easy to do this as long as you keep it to simple float data like a drone gain.

Another important thing you must add to the fluid design is a function called void idle(). In this function add a code block that has these 2 lines:
Fl::check();
Fl::flush();


To help clarify everything here's a screenshot of ntk-fluid once I've done all this. Its actually a pretty good overview what we've done so far:


Possibly the biggest departure from what we've done previously is now the program will not be a stand-alone binary, but a library that has functions to get called by the host (just like in the DSP). This means some major changes in our stuck_ui_main.cxx code.

For the GUI the most important functions are the instantiation, cleanup, and port event. To use NTK/fltk/ffffltk you will need to use some lv2 extensions requiring another function called extension_data but we'll discuss it later. The instantiation is obviously where you create your window or widget and pass it back to the host, cleanup deallocates it, and the port event lets you update the GUI if the host changes a port (typically with automation). We'll present them here in reverse order since the instantiation with NTK ends up being the most complex. So port event is fairly straightforward:


void stuckUI_port_event(LV2UI_Handle ui, uint32_t port_index, uint32_t buffer_size, uint32_t format, const void * buffer)
{
    StuckUI *self = (StuckUI*)ui;
    if(!format)
    {
      float val = *(float*)buffer;
      switch(port_index)
      {
        case STICKIT:
          self->stickit->value((int)val);
      self->led->value((int)val);
      break;
        case DRONEGAIN:
          self->volume->value(val);
      break;
        case RELEASE:
          self->time->value(val);
      break;
      }
    }
}


The enlightening thing about doing a UI is that you get to see both sides of what the LV2 functions do. So just like in the widget callbacks you send a value through the write_function, this is like what the write function does on the other side, first you recast the handle as your UI object so you can access what you need, then make sure its passing the format you expect (0 for float, remember?). Then assign the data corresponding to the index to whatever the value is. This keeps your UI in sync if the host changes a value. Nice and easy.

Next up is the simplest: Cleanup:
void cleanup_stuckUI(LV2UI_Handle ui)
{
    StuckUI *self = (StuckUI*)ui;

    delete self;
}






No explanation necessary. So that leaves us with instantiation. This one is complex enough I'll give it to you piece by piece. So first off is the setup, checking that we have the right plugin (this is useful when you have a whole bundle of plugins sharing code), then dynamically allocating a UI object that will get returned as the plugin handle that all the other functions use, and declaring a few variables we'll need temporarily:
static LV2UI_Handle init_stuckUI(const struct _LV2UI_Descriptor * descriptor,
        const char * plugin_uri,
        const char * bundle_path,
        LV2UI_Write_Function write_function,
        LV2UI_Controller controller,
        LV2UI_Widget * widget,
        const LV2_Feature * const * features)
{
    if(strcmp(plugin_uri, STUCK_URI) != 0)
    {
        return 0;
    }

    StuckUI* self = new StuckUI();
    if(!self) return 0;
    LV2UI_Resize* resize = NULL;


Then we save the write_function and controller that got passed in from the host so that our widgets can use them in thier callbacks:
    self->controller = controller;
    self->write_function = write_function;


Next stop: checking features the host has. This is where using NTK makes it a bit more complicated. The host should pass in a handle for a parent window and we will be "embedding" our window into the parent. Another feature we will be hoping the host has is a resize feature that lets us tell the host what size the window for our plugin should be. So we cycle through the features and when one of them matches what we're looking for we temporarily store the data associated with that feature as necessary:
    void* parentXwindow = 0;
    for (int i = 0; features[i]; ++i)
    {
        if (!strcmp(features[i]->URI, LV2_UI__parent))
    {
           parentXwindow = features[i]->data;
        }
    else if (!strcmp(features[i]->URI, LV2_UI__resize))
    {
           resize = (LV2UI_Resize*)features[i]->data;
        }
    }


Now we go ahead and startup our UI window, call the resize function with our UI's width and height as arguments and call a special NTK function called fl_embed() to set our window into the parent window. It seems this function was created specially for NTK. I haven't found it in the fltk source or documentation so I really don't know much about it or how you'd do it using fltk instead of NTK. But it works. (You can see the NTK source and just copy that function). EDIT: one important detail that I missed is that you are supposed to fill in the LV2UI_Widget that the host passes with your UI widget. When your UI is x11 based you pass in the xid from the x window or at least set it to zero. This is done below after fl_embed().  Once that's done we return our instance of the plugin UI object:
    self->ui = self->show();
    fl_open_display();
    // set host to change size of the window
    if (resize)
    {
       resize->ui_resize(resize->handle, self->ui->w(), self->ui->h());
    }
    fl_embed( self->ui,(Window)parentXwindow);

    *widget = (LV2UI_Widget)fl_xid(self->ui);


    return (LV2UI_Handle)self;
}


Ok. Any survivors? No? Well I'll just keep talking to myself then. We mentioned the extension_data function. This function gets called and can do various special functions if the host supports them. Similar to the port event, the same extension_data function gets called with different indexed functions and we can return a pointer to a function that does what we want when an extension we care about gets called. Once again we get to see both sides of a function we called. The resize stuff we did in instantiate can be used as a host feature like we did before or as extension data. As extension data you can resize your UI object according to whatever size the host requests. This extension isn't necessary for an NTK GUI but since the parent window we embedded our UI into is a basic X window, its not going to know to call our fltk resize functions when its resized.

In contrast, a crucial extension for an NTK GUI is the idle function. Because similarly the X window doesn't know anything about fltk and will never ask it to redraw when something changes. So this LV2 extension exists for the host to call a function that will check if something needs to get updated and redrawn on the screen. We made an idle function already to call in our StuckUI object through fluid, but we need to set up the stuff to call it. Our extension_data function will need some local functions to call:
static int
idle(LV2UI_Handle handle)
{
  StuckUI* self = (StuckUI*)handle;
  self->idle();
 
  return 0;
}

static int
resize_func(LV2UI_Feature_Handle handle, int w, int h)
{
  StuckUI* self = (StuckUI*)handle;
  self->ui->size(w,h);
 
  return 0;
}



Hopefully its obvious what they are doing. The LV2 spec has some stucts that are designed to interface  between these functions and the extension_data function, so we declare those structs as static constants, outside of any function, with pointers to the local functions :
static const LV2UI_Idle_Interface idle_iface = { idle };
static const LV2UI_Resize resize_ui = { 0, resize_func };


And now we are finally ready to see the extension_data function:
static const void*
extension_data(const char* uri)
{
  if (!strcmp(uri, LV2_UI__idleInterface))
  {
    return &idle_iface;
  }
  if (!strcmp(uri, LV2_UI__resize))
  {
    return &resize_ui;
  }
  return NULL;
}

You see we just check the URI to know if the host is calling the extension_data function for an extension that we care about. Then if it is we pass back the struct corresponding to that extension. The host will know how these structs are formed and use them to call the functions to redraw or resize our GUI when it thinks its necessary. We aren't really guaranteed timing for these but most hosts are gracious enough to call it at a frequency that gives pretty smooth operation. Thanks hosts!


So, its now time for the ugly truth to rear its head. Full disclosure: this implementation of the resizing extension code doesn't work at all. The official documentation describes this feature as being 2 way, host to plugin or plugin to host. We've already used it as plugin to host and that works perfectly, but when trying to go the other way I can't get it to work. The trouble is when we declare and initialize the LV2UI_Resize object. The first member of the struct is type LV2UI_Feature_Handle which is really just a void* which should really just be a pointer to whatever data the plugin will want to use when the function in the 2nd member of the struct gets called. Well for us when resize_func gets called we want our instance of the StuckUI that we created in init_stuckUI(). That would allow us to call the resize function. But we can't because its out of scope, and the struct must be a constant so it can't be assigned in the instantiate function. So I just have a 0 as that first argument and actually have the call to size() commented out.

Perhaps there's a way to do it, but I can't figure it out. I included that information because I hope to figure out how and someday make my UI completely resizable. The best way to find out, I figure, is to post fallacious information on the Internet and pretty soon those commenters will come tell me how wrong and stupid I am. Then I can fix it.


As a workaround you can put in your manifest.ttl this line:
lv2:optionalFeature ui:noUserResize ;

Which will at least make it not stupidly sit there the same size all the time even when the window is resized. If the host supports it.

EDIT: I understand that returning the correct LV2UI_Widget from instantiate should allow the plugin to resize without using the resize extension. It also allows for keyboard entry or modifiers.  Then the workaround is unnecessary.

"So if its not even resizable why in the world did you drag us through 3 long detailed posts on how to make LV2  GUIs out of SCALABLE vector graphics?!" you ask. Well, you can still make perfectly scalable guis for standalone programs, and just having a WYSIWYG method of customized UI design is hopefully worth something to you. It is to me, though I really hope to make it resizable soon. It will be nice to be able to enlarge a UI and see all the pretty details, then as you get familiar with it shrink it down so you can just use the controls without needing to read the text. Its all about screen real estate. And tiling window managers for me.


So importantlyin LV2 we need to have a standard function that passes to the host all these functions so the host can call them as necessary. Similar to the DSP side you declare a descriptor which is really a standard struct that has the URI and function pointers to everything:
static const LV2UI_Descriptor stuckUI_descriptor = {
    STUCKUI_URI,
    init_stuckUI,
    cleanup_stuckUI,
    stuckUI_port_event,
    extension_data
};


And lastly the function that passes it back. Its form seems silly for a single plugin, but once again you can have a plugin bundle (or a bundle of UIs) sharing source that passes the correct descriptor for whichever plugin is requested (by index). It looks like this:
LV2_SYMBOL_EXPORT
const LV2UI_Descriptor* lv2ui_descriptor(uint32_t index)
{
    switch (index) {
    case 0:
        return &stuckUI_descriptor;
    default:
        return NULL;
    }
}


As a quick recap, here are the steps to go from Inkscape to Carla (or your favorite LV2 plugin host):
1. Draw a Gui in Inkscape
2. Save the widgets as separate svg files
3. Convert to cairo code header files
4. Edit the draw functions to animate dials, buttons, etc. as necessary.
5. Create the GUI in ntk-fluid with the widgets placed according to your inkscape drawing
6. Include the ffffltk.h and use ffffltk:: widgets
7. Assign them their respective draw_functions() and callbacks
8. Add the write_function, controller members, and the idle() function
9. Export the source files from fluid and write a ui_main.cxx
10. Update your ttl
11. Compile, install, and load in your favorite host.

Our plugin in Jalv.gtk


So you now have the know-how to create your own LV2 plugin GUIs using Inkscape, svg2cairo, ffffltk, ntk-fluid, and your favorite editor. In 11 "easy" steps. You can see the source for the infamous Stuck that I developed this workflow through in my infamous repository. And soon all the plugins will be ffffltk examples. I'll probably refine the process and maybe I'll post about it. Feel free to ask questions. I'll answer to the best of my ability. Enjoy and good luck.

As an aside, in order to do this project. I ended up switching build systems. Qmake worked well, but I mostly just copied the script from Rui's synthv1 source and edited it for each plugin. Once I started needing to customize it more to generate separate dsp and ui binaries I had a hard time. I mostly arbitrarily decided to go with cmake. The fact that drmr had a great cmake file to start from was a big plus. And the example waf file I saw freaked me out so I didn't use waf. I guess I don't know python as much as I thought. Cmake seemed more like a functional programming language, even if it is a new syntax. I was surprised that in more or less a day I was able to get cmake doing exactly what I wanted. I had to fight with it to get it to install where I wanted (read: obstinate learner), but now its ready for whatever plugins I can throw at it. So that's what I'm going to use going forward. I'll probably leave the .pro files for qmake so if you want to build without a GUI you can. But maybe I won't. Complain loudly in the comments if you have an opinion.

No comments: