20 May 2015

More on syntax checking vala - and a nice video

As I wrote last week. I had added full syntax checking to the editor. So it runs a full compile check as you type.
Here's a nice video of it working...

After the initial joy of adding this to code, I soon realized it had a fatal flaw, read on to find out more..

Posted by in Gtk | Add / View Comments()

09 May 2015

Fetching Resources from github in the App Builder and fake web servers

My final words this week on the builder - handling resources, and fake web servers

While I talked in the other posts about how the builder extracts the API for various components from the libvala library and the vapi files, some information that the builder requires has to be manually, created or fetched from other locations.

When the Builder was written in seed, it basically looked at the source code directory, and read files relative to the source code. For the Vala version however, it's not expected to know about the source code directory, so I had to use a different approach.

Posted by in Gtk | Add / View Comments()

08 May 2015

libvala testing code and extracting API from the vapis

And the next part in the series. Gir and Vala structures, Nothing like a slow day to write a few blog posts. 

The App Builder was originally designed to build applications using seed (the gobject introspection webkit javascript engines bindings), One of the key elements of how this was done involved introspecting the Gtk API, and extracting all the properties, signals and class structure.

In this post I will go through the history of how I extracted the API information on Gtk, initially from Gobject introspection and GIR files, upto the current version which uses libvala to get the correct API direct from the vapi files.

Posted by in Gtk | Add / View Comments()

07 May 2015

App Builder - Database based Plugin builders for Web components.

It's been a busy month, unfortunately not for our paid work, which has dropped down to a trickle. Taking advantage of this I've been building more into our App Builder. This post hopefully is the first in a series about some of those additions.
The Primary purposes of our Builder is
  • A WYSIWYG tool for web applications using both Bootstrap or the RooJS libraries.
  • A new visual way of building Gnome/Gtk Applications 
In working towards these goals the builder has moved forward in a few directions. the first one that this blog post talks about is generating User interfaces from Database Schemas.

Posted by in Gtk | Add / View Comments()

25 Apr 2011

Gtk3 introspection updates and Unusable Unity..

Well, as Gnome 3 is out, it has to be tested. Luckily I've not got a huge deployment to sort out, but as I have a few applications that use Gtk, I thought it was about time I upgraded one of my machines to see what chaos I will have to deal with in the future.

So it was one of my Ubuntu boxes that got the pleasure of a Natty and Gnome3 PPA upgrade. (I use debian on my other development box, which actually got destroyed last week with a complete disk failure, although I suspect the motherboard may have problems... It's getting old like me...)

Upgrading to Natty is not to bad, from what I remember it only took a small amount of brain surgery to get it to boot correctly after the upgrade. But once up, you get the pleasure of the Unity desktop. My first impressions where not to hot on unity, my wife uses it on her netbook, it's great there, after the initial shock of me upgrading without her knowing, she actually said it was alot better than compiz. Although she missed the special effects.

But after using Unity on the big screens, it just became unbearable. Detached menus may seem like a cool idea, and are quite handy on a netbook, but they are an absolute nightmare when using things like gimp on dual head full HD monitors, my wrists hurt after a few minutes.... 

Along with the removal of the Applications/Places/System menu's which while klunky are still handy for quickly finding applications. A classic example of this Alleyoop Memory Checker, a very nice wrapper around valgrind. In the Unity world if you do not know the name of the application, then finding it is a huge mouse journey around big icons. 

As for the left icon menu, all I can say is that I'm not the worlds best designer (although at least I did study it), but it's so graphically noisy that it unusable. It's basically a bad re-invention of Docky/Cairo Dock, which do far better jobs at providing a similar task role. 

So after all that I did try and get gnome-shell going, but unfortunatly the Gnome3 PPA build is not currrently working, and also has a rather nasty habit removing all usable desktop enviroments. I ended up adding xterm to one of the /etc/Xorg/X.sessiond files and starting up gnome-panel, mutter and docky to produce a usable desktop for the time being, while I wait to test out the latest gnome-shell.

So on with the harder stuff.. - Gtk3 and introspection.

One of the key applications I use to develop is app.Builder.js , it's a drag/drop interface to build web applications, that also allows you to fill in all the code and associate it clearly with the element and event occuring. It's written in Javascript, and uses Gnome seed to run on the desktop. As I've mentioned before Seed is a bridge layer between the Webkit Javascript engine, and Gobject-introspection, the now standard way to interface Gnome/Gtk/Glib etc. projects to non-C languages, eg. Python, Javascript (and others...)

With the introduction of Gtk3, GObject introspection has also been updated, and the updated mix between the two had quite a few knock on effects to the builder I had written using Gtk2 and pre-0.9 versions of Gobject introspections. Heres a general summary of the changes.

TreeIter and TextIter 

The latest version of GObject introspection has a feature called caller allocates, this basically means that previously with Seed we had to create an instance of a TreeIter, the the TreeIter call would be of type 'inout' (eg. the Iter would be sent into the method, and returned out)

var iter = new Gtk.TreeIter();
model.get_iter_from_string(iter, path);
// iter would now contain the tree iter for that path..

In newer versions, the iter is an 'out' value, which means you have to create an object for the iter to be added to. eg.
var iret = {};
model.get_iter_from_string(iret, path);
// iret.iter now contains the tree iter.


since the get_selected method for a GtkTreeSelection now has 2 out values, the call has change from

var iter = new Gtk.TreeIter();
selection.get_selected(model, iter);

var sret = {};
// sret now contains { model: **THE MODEL**, iter: **THE ITER** }

TreeModel get_value

Since get_value does not have a return value, seed with return the 'out' values as the return object.

var value = new GObject.Value('');
model.get_value(iter, 2, value);

var str = model.get_value(iter, 2).value.get_string();

Drag pixmap becomes surfaces

This is a pure Gtk3 API change (BC break)

var pix = widget.create_row_drag_icon ( path);
Gtk.drag_set_icon_pixmap (ctx, pix.get_colormap(),   pix,  null, ..... )

var pix = widget.create_row_drag_icon ( path);
Gtk.drag_set_icon_surface(ctx, pix);

Drag drop data passing..

The drag drop signals appear to work ok, however I've not managed to get the data to go back and forth, 
a quick workaround is to just use some form of global variable to store the current dragged item (I doubt you will get more than one dragged item at once..)

Drag drop API

alot of these appear to have played musical chairs.
GtkWidget.prototype.drag_source_set -> Gtk.drag_source_set
Gtk.drag_source_set_target_list -> GtkWidget.prototype.drag_source_set_target_list
Gtk.drag_dest_set -> GtkWidget.prototype.drag_dest_set

Internal Seed changes

I've added a few more fixes to Seed in the last few weeks, mostly to handle compiling correctly and detecting the correct version of introspection. for the most part it's working fine, however I'm still a bit baffled by a Glib memory corruption bug, which occured after multiple model.set_value and model.get_value calls. After running valgrind, I managed to stop the corruption occuring by increasing the allocated size for a struct by 1 byte 
Around line 546 and 640 of seed-engine.c the change goes something like this.
-                  out_args[n_out_args].v_pointer = g_malloc0 (size);
+                  out_args[n_out_args].v_pointer = g_malloc0 (size+ 1);

Arround line 738 of seed-structs.c  
-  object = g_slice_alloc0 (size);
+  object = g_slice_alloc0 (size +1);

Posted by in Gtk | Add / View Comments()

07 Aug 2007

GtkDjs, daily builds, added libraries and much more..

Daily builds / downloads available

I've set up a cron job now, so all the bindings and binaries are built daily, you can download them from here.
This includes gtkjs, mjs and jsfastcgi, In theory it would not take too much to build this on Windows... - Send me the fixes if you attempt it...

Just download, unzip and try out
./gtkjs test/filetree.ds

the gtkdjs Documentation is updated daily to match

And I still need a better name for the project;)

Closure and scoping fixes.

After seeing a post on the DMDScript newsgroup concerning closures and scoping issues, I spent quite a while looking at the issue.

The basic problem, was that the original DMDscript code was designed to create a cache of function definitions, and re-use them whenever they where refered to. Along with this, they had no concept of creationScope. so variables in the creating scope where not available. Fixing this involved quite a few changes to the compiler and runtime. Basically creating new Function instances when they are found, rather than only once at compile time. Along with storing the scope inside the Function instance. so when they are called they understand the correct scope. Anyway standard Javascript scoping, and closures all work as expected.

Undefined Warnings improved

I mentioned in my last post that I added Warnings when undefined variables where accessed, I've modified this slightly as undefined is a valid type in Javascript, so unassigned arguments to a function call are flagged as undefined, but accessing them should not issue a warning. So only when the scope.Get(variablename) calls return null, rather than a Value.vtype[V_UNDEFINED] flag this warning now.

LibSoup extension

LibSoup provides HTTP support, at present, basic sync requests work, the code is there for async calls, but I've not tested it yet. test code is in tests/Soup/.

This little bit of code get's my web site, and shows headers, then extracts all thehref from all the <a> tags.
var sess = new Soup.SessionSync();

var msg = new Soup.Message("GET", "");
status = sess.sendMessage(msg);

println("Status is " + status);
println("ResponseHeaders is " + msg.responseHeaders.toSource());

var imp = new Dom.Implementation;
var doc = imp.createHtmlDocFromMemory(msg.response);
var a_s = doc.getElementsByTagName("a");
for(var i =0; i < a_s.length; i++) {
println("got href: " + a_s.item(i).getAttribute("href"));

XML2 limited support

To enable parsing of html documents for Gdome, I've added limited support for XML2, it's mostly used by the Gdome method: Dom.Implementation.prototype.createHtmlDocFromMemory( String htmltext )
The Generator code is currently tied into the libxml header files on my system.. this needs fixing so it uses the gtkwrap download script. (but the code is a nice example of how to automatically create bindings based on .h files..)

Gtk.SourceView support

In preperation for testing text editing (and eventually self-editing / morphing of applications), I've bound the GtkSourceView widget. Unlike the standard Leds editor which uses scintilla, I thought I'd try and see if using GtkSourceView would prove more robust. - The bindings had already been done for GtkD, so there was not much to change to the APILookupSourceView.txt code. I only used the full name "sourceview" rather than "gsv"

Handling of missing functions and Libraries improved

When I was testing GtkToolbar on one of my machines which had an old Gtk library, I discovered a segfault when it called a newer function that did not exist in the old library. To fix this, the Loader.d file now binds a simple function that throws and exception to any method it can not bind. So you can catch in javascript any call to unsupported methods in a library.

Phobos Path module added

all the methods from std.path are now exported to javascript, and in the manual. This enables a few usefull functions like

var file = "/etc/passwd"
println("Directory = " + Path.getDirName(file));
println("Filename = " + Path.getBaseName(file));
println("combined filename = " + Path.join("etc","passwd"));

Regex Fixes

In testing extjs parsing, I came across two bugs in D's regex implementation, that prevented it's loading.
  • forgetful parsing (?:abc|cde|xyz) - the fix for this was trivial and is in D's bugtracker, and there is a modified regex file in the dmdscript distribution.
  • the use of special chars in character range matching after the '-'. eg. [\w-\*] and [\w-\.] which should be interpreted as [\w\*-] and [\w\.-]. These currently parse correctly however I've not got round to fixing the implementation properly so they are pretty broken at present. (basically matching everything from \w onwards...) - the bugs in D's bugtraker, but the fix is not... I may get back to this...

Simplified exporting of Structures

One of the missing pieces for the Gtk Bindings, that I came accross with jLeds, was the event structures had not been bound. Since they are very simple structs, and you only need to get access to their properties, I developed a new action for the wrap file "structDump" which autogenerates the class file, forces the full Struct definition to be written and writes getter methods for all the exportable properties.

Bindings builder flexibility

Since we now have about 20 libraries bound, the Wrapping builder script can take a little while on slow systems to build all the bindings code. So I've added the ability to specify which APILookup files to action.
eg. to build LibSoup you can do.
sh APILookupGLib.txt APILookupGObject.txt APILookupSoup.txt 
It cant do the dependancy resolution yet, so you have to list all the pre-requisite packages so it has access to all the required types when building the library you need.

The struct parser is also a little more robust now, handling comments and lists of elements eg.
ushort x, y;

Minor Gtk Fixes

  • G.Object.setProperty(String key, Any value) now works, and converts value into G.Values automatically.
  • GtkTreePath[] Gtk.TreeSelection.prototype.getSelectedRows(Gtk.TreeModel model) now works - although I need to look at generic ways to generate array of Objects from GList..
  • new Gtk.TreeStore(Array coltypes) now works (where coltypes is either an array of strings, or G.Types.) - this works along with the other constructor methods - eg. varargs......
  • Gtk.TreeStore.prototype.set(Gtk.TreeIter row, Array values) now works to quickly set the values of a tree row, rather than calling set(GtkTreeIter row, Number col, Mixed value)
  • Gtk.Container.prototype.remove(Number item) and Gtk.Container.prototype.remove(Gtk.Widget widget) are now working - I need to change this slightly, so you can remove all by sending '0' as the item, and waiting until it returns false...
  • Most ofthe GtkEvent Structures are now exported, and all the properties are available as getters.
  • new GdkColor(Number color) is based on 0xRRGGBB hexidecimal values, rather than longintegers.. (you can use the long version to do more precise colours: new GdkColor(Number red, Number green , Number blue);
  • new Gdk.Pixbuf(Array xpmdata) is now supported so converting xpm data for use as icons is very simple (just replace "const char**" with "var")

ExtJs loading

One of the goals is to make the engine compatible with existing code out there, a nice test of this was extjs, which it can now load correctly (with a few lines of javascript prefixing the loading).I've not actually tested any of the functionality yet, as XmlHttpRequest needs building from the libsoup bindings.

jLeds port underway..

What's a language without an editor (or an email client - if you know the old joke about every application eventually evolves until it can read email.).. So partly to test the bindings I've started looking at porting Leds to gtkjs - the code is in tests/jLeds, and gives a good example of some of the new Javascript2 features, along with the gtk bindings.

Try the newsgroup if you need support.

Posted by in Gtk | Add / View Comments()

31 Jul 2007

gtkds - more updates and better debugging..

Add one feature, and create a few bugs.. seems like it's always the way..

The two features added over the weekend where class syntax support and include support. My initial effort proved to be a little off the mark.

Read on if you want to know the nitty gritty details about writing an interperted language runtime... Along with a list of the Gtk binding improvements..

Posted by in Gtk | Add / View Comments()

28 Jul 2007

Compairing Adobe AIR to dmdscript / fastcgijs / gtkjs

A good friend of mine asked what's the difference between the dmdscript/fastcgijs/gtkjs stuff I'm doing and Adobe AIR.

I've not really looked at Adobe AIR much, noticed that it was mentioned on the extjs site, so I thought I'd do a quick "what's the difference"..

So what's the key differences

For Adobe AIR
  • Backed by a huge company with unknown reasons for hooking you in.
  • Lot's of support for Adobe technology, but any other libraries, you can forget about.
  • Probably written in C++ or C, so there are probably a few nice security holes difficult to find in the memory allocation. Even if you did have access to the source - it would be a pretty complex task to write bindings for libraries.
  • Great support for Windows, pretty good for Mac, and bugger all for other un*x's
For GtkJs / Fastcgijs / dmdscript
  • Absolutly no support ;) - obviously only done, because it's interesting and fun!
  • Lots of support for any open source library, as long as you are willing to waste a few days working out how mindblowingly simple it is to write bindings.
  • Written in D, which makes it pretty secure (well if you fix some of the bugs in my generator to do a small amount of sanity checks). And due to the fact that it's so similar in syntax to Javascript that makes the whole process of moving slow parts into compiled code considerable simpler than any other scripted language..
  • Great support for linux, and it might just work on any other platform, as long as you can find someone who knows those platforms.
Personally I think the Adobe AIR idea is pretty good, Javascript is a good basis for a language. But to be honest for the reasons above, I would not touch it with a barge pole..
Posted by in Gtk | Add / View Comments()

27 Jul 2007

javascript2, ECMAScript4 and dmdscript.

Last week I had a go using my little bindings for something useful, rather than just being a little toy. One of the projects I was working on was a hack to exim4, that logged all incomming and outgoing attachments into mysql and stored them on the filesystem.

The basic idea was that the company needed the ability to review if any sensitive data was being sent out by email. The exim hacks are pretty simple for this (hook into the mime handler). The web viewer uses extjs and PHP at the backend, it's a small addition to their current email management interface.

But one area that was needed was a little cron job that cleaned up the directories and deleted data and files older than 2 weeks (so they dont get too clogged up.)

I thought this would be an interesting test of the mysql and file system bindings. So I threw together a horrifically simplistic version of DataObjects, and a short script that did an SQL query and based on the results deleted records and files.

One issue was that I could not really do the DataObjects thing of overlaying the results, or query options into the dataobject as object variables in javascript are overlaid into the same namespaces as methods, so you could accidentally delete methods if your database had matching names. - So the data just gets put in (dataobject).data.*

The other issue was that the syntax for javascript while fun and flexible could really do with a class construct. If you look through the specifications for ECMAScript4, There is a huge amount of changes making Javascript as we know it far more Java / PHP5'y.. with interfaces, public private etc. So I had a go at adding some of the new features into the dmdscript core.

I've managed to get class's and include working. although it probably needs quite a bit more testing. This basic syntax now works
include "mytest.ds"; // this is run at compile time (not runtime!)

class A {
function A() {
println("This is the constructor A");

var c = 12;
function B() {
println("This is B and c is " + this.c);

class C extends A
function C() {
println("This is the constructor C");;
function D() {
println("This is D and c is " + this.c);

var a = new A();

var c = new C();

Will successfully output

Hello from mytest
This is the constructor A
This is B and c is 12
This is the constructor C
This is the constructor A
This is B and c is 12
This is D and c is 12

Now I can start writing tidy little javascript libraries...

Posted by in Gtk | Add / View Comments()

13 Jul 2007

dmdscript with fastcgi

I've been slowly ticking away adding features to the gtk javascript bindings, in doing so, I'm adding extra libraries - Mysql works quite well, and Seeing the recent post on about fastcgi4d, I wondered if I could run dmdscript from fastcgi.

To enable a web version of dmdscript with my bindings, I had to do quite a bit of re-organization, enabling it to be built with random libraries added in and exclude the Gtk stuff. Core to this was to move all the registration code into the directories that hold the binding code. Each binding directory now contains 3 files, binding***.d, binding***type.d and register***.d. Which manage the dynamic loading of .so (or .dll if windows actually works). and the registering of all the javascript objects and methods.

For fastcgi, I had a look at the code described on the blog post, but was very reluctant to use it, as it
  • required tango, which is not a current requirement, and I'm a little concerned about using.
  • required knowlege of templates, which are still a bit of a black art. and I've only used very sparingly when absolutly every other alternative has been ruled out.
So after considerable digging through the code and trying to make head or tail of the code on, I finally found the python fastcgi code, which ported easily to D.

The core files libraries that are helpers in dealing with fastcgi are here.

They need the loader.d and paths.d file from here
to be usable.

and the current, non-threaded simple responder is here

So for hello world example, this simple piece of code dumps all the server variables and says hello world.

println("hellow world from javascript");
Next job is to look at how GET/POST data is passed around and how to escape data on output. among a list of 100 ideas for how it could all work.
Posted by in Gtk | Add / View Comments()
   (Page 1 of 2, totalling 12 entries)    next page »

Follow us on