11 Apr 2007

making XSLT readable - has this been done before?

We where discussing with one of my clients the idea of including dynamic content on their home page, which is currently static HTML, due to extreme load requirements. Normally Php is only reserved for specific parts of the site, and not on the front page.

I had the idea of using XmlHttpRequest, however another member of the team suggested XSLT, and I have to admit, I've never seen that idea before, and the example he showed basically included content from an external file by just adding a simple tag like
<our:include src="mydynamic.html" part="xxx" />
Where the mydynamic.html could use the 404 handler trick that php.net uses. (eg. if the file doesnt exist, it gets generated by the 404 handler, and deleted by a cronjob every 5 minutes etc.)

My only reservation about this is really that XSLT templates are unreadable garbage. Hence if we ever started to get clever with them, they could quickly become unmaintainable.

This however didnt seem to be a problem so much with XSLT but the fact that reading and writing XSLT files is not particulaly simple. Looking at the Basic specification for 1.0, and the example file he had, it seem clear that this file could easily be generated by a simple tool. However I could not find anything after a quick google search that seemed to exist to do this. So I threw together a simple proof of concept.

What if the template looked more like this:
template("/") {
applyTemplates;
}

template("*") {
copy {
applyTemplates("@*|node()");
}
}

template("processing-instruction()") {
copy {
applyTemplates("@*|node()|text");
}
}

// make h1 green

template("html:h1")
{
copy {
attribute("style") <colour:green>
applyTemplates("@*|node()");
}
}

template("our:include")
{
variable("part", "@part");
applyTemplates("document(@src)//*[@id=$part]");
}


Writing a simple parser generator in D, enabled the generation of the xslt file from this file.

Compile parser generator:
dmd xsltgen.d -of/tmp/xsltgen -od/tmp

Convert the above file to xslt:
/tmp/xsltgen inctest.xslts

And magically you have a generated file, and a maintainable version.. - Makes me wonder though, has someone done this already?

the source: xsltgen.d
output from the above example: inctest.xslt

25 Mar 2007

Javascript Tookits Part Two

<style type="text/css"> <!-- #toc, .toc, .mw-warning { border: 1px solid #aaa; background-color: #f9f9f9; padding: 5px; font-size: 95%; } #toc h2, .toc h2 { display: inline; border: none; padding: 0; font-size: 100%; font-weight: bold; } #toc #toctitle, .toc #toctitle, #toc .toctitle, .toc .toctitle { text-align: center; } #toc ul, .toc ul { list-style-type: none; list-style-image: none; margin-left: 0; padding-left: 0; text-align: left; } #toc ul ul, .toc ul ul { margin: 0 0 0 2em; } #toc .toctoggle, .toc .toctoggle { font-size: 94%; }@media print, projection, embossed { body { padding-top:1in; padding-bottom:1in; padding-left:1in; padding-right:1in; } } body { font-family:'Times New Roman'; color:#000000; widows:2; font-style:normal; text-indent:0in; font-variant:normal; font-size:12pt; text-decoration:none; font-weight:normal; text-align:left; } table { } td { border-collapse:collapse; text-align:left; vertical-align:top; } p, h1, h2, h3, li { color:#000000; font-family:'Times New Roman'; font-size:12pt; text-align:left; vertical-align:normal; } --> </style> <div> <p>My last post about Javascript toolkits garnered a few comments, which seems to be rare for most of my blog posts... (I must be getting too old..or the spam protection works to well). There where quite a few suggestions for alternatives to dojo, along with a not too complimentary comment about dojo's development status. </p> <p /> <p>As luck had it, I started a new project Last week, a contact manager for one of my clients, and rather than carry on with dojo, that had begun to annoy me a little, with little bugs (for example , the Accordion was broken), or just weird stuff, like getting forms on dialogs to look just right. I thought that it would be a good opportunity to review some of the other toolkit's mentioned on the post.</p> <p /> <p>The extended Entry contains reviews of jQuery and yui-ext (and see which is now the prefered library)</p> </div> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/133/Javascript_Tookits_Part_Two.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/272/Javascript.html">Javascript</A> | <A href="/blog.php/View/133/Javascript Tookits Part Two.html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">08 Mar 2007</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/132/Javascript_Toolkits.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">Javascript Toolkits</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"> <p>Like alot of people, I'm really impressed with google's applications. They have created web applications that behave alot like desktop ones. Core to this is the javascript tricks that hold it together.<br /><br />In the past few years I've been building applications using XUL, that provides a similar set of functionality, and is pretty fast to develop with. However we have run into a few issues<br /></p><ul><li>Its not cross browser, which is ok for intranet style applications, obviously is a little problematic for public internet ones.</li><li>After many projects we began to see that dialog, modal prompts where better for user input, as the prevent the user modifying the application state while entering data, along with making it less likely to forget to press the submit button. Recent builds of Firefox have disabled the modal feature of popup window, and along with the noticeable speed penalty on most modern operating systems for opening new windows, these issues begins to dig into the benefits of this approach.</li><li>WYSIWG editors are still not easy and simple to use...<br /></li></ul><p><br />Since one of my pet projects, an online accounting package needed a bit of sprucing up, and testing my home grown JavaScript code on IE is such a nightmare, I started looking at JavaScript toolkits.<br /><br />The biggy from countless blog posts recently has been YUI, Yahoo's interface toolkit. I spent quite a while looking at this, the website for it is very complete, along with a comprehensive set of widgets. There was alot too like about it, however as I dug deeper, I began to dislike some of the things i saw.<br /><br /></p><ul><li>silly things like all the JavaScript code libraries appeared to use a weird, double lined coding standard.<span style="font-style: italic;">yes I know it's silly, but it makes a difference when you read code to see something that odd.</span></li><li>(from what i remember ), alot of those set*, get* methods.</li><li>almost all the demos appeared to create the widgets in JavaScript and using js associative arrays to configure the widgets , rather than using extra attributes on existing html tags, and doing some kind auto conversion.</li><li>the constructors did some strange namespace hack.</li></ul><p>In general, it looked like using the widgets was very un-javascripty.<br /><br />So after some consideration, I started looking for alternatives. The first I came across was prototype which looked more like someone rewriting JavaScript than a toolkit, then i came across <a href="http://www.dojotoolkit.org/">dojo</a>, <br />which looked alot more promising. The website has focused on the demos, which is pretty good. There is documentation (i'll talk about that more later)</p><p>Best of all it works alot more like the widget based JavaScript code i've written. it focuses on converting existing dull html widgets into super js enabled ones at the flip of a switch.<br /><br /></p><h3>Concept</h3><p>Conceptially, as show on the demo's, takes the HTML page, and when the library is loaded, it hooks into the onload method of the page, and looks for all the widgets with the attribute dojotype="Select" or similar, and has a javascript class to handle each "dojotype". <br /></p><p>The dojo widget class takes a look at the attributes, and sets the javascript object properties to match those HTML attributes (hence behaving like the configuration for the widget). Then it builds the new widget based on those attributes, and merges it in with a template file (which is in dojo/src/widgets/templates/*) or uses a text attribute of the javascript object for the template. It then adds all the behaviour and just makes stuff happen.</p>This means getting upto speed with the widget set is fast, and you only slow down when you want to alter the default behaviours, at which point looking through the code seems to be the best idea.<br /><br />Unfortunatly, I found the documentation for dojo, a little thick and rather pointless, it comes of as more of a overview, and the real information is so far down in the chapters that it doesnt help much.. Rather than a nice reference of all the attributes that each widget type uses. (although the API reference is quite handy)<br /><br /><h3>Features</h3><p>Dojo has all the basic features that you would expect from a javascript widget toolkit, dialogs, windows, autocompletion, clever select widgets and more. It also has quite a library of backend tools for formating dates, doing xmlhttprequests. and more. What is nice about this, is that these libraries are not forced on you, rather they are used by the widget libraries, and you dont really need to know about them, or learn them. And offer a gradual learning path. So when you are ready you can start diving in and seeing if they are usefull in other parts of your code.</p>The architecture looks to be extendable as well, so you theoretically could build on the existing widget types and create your own bastard children, suiting your needs better. The syntax for the widget's is realitively straight forward, although they do use a slightly strange javascript syntax, but I doubt there is a better way to do what dojo acheives.<br /><p>Most of the really usefull stuff is in src/widgets/*<br /></p><h3>Flaws</h3><p>Well, nothings ever perfect, and looking at the huge number of bugs in the dojo bugtracker, I suspect they developers are having difficulting keeping up with it's popularity. <br /><br />There are a few design decisions that where annoying (although this has to be taken against the fact that the majority where excellent). The key ones being:</p><ul><li>CSS interaction, Widget's can load their own CSS from the template directory, This makes modifying the CSS for widget's rather difficult. (unless you treat the template directory as your own working space - or override it somehow). The more I worked with it, the more I thought it would be better to provide a generic CSS file that contains all the CSS needed for the widgets (or a file that @imports all the individual ones) , and force you to load it manually. - hence allowing you to replace the CSS quickly and simply. As often the dojo styling can conflict with any generic CSS that you may have already for your application.<br /><br />In addition, there are quite a few style="..." tags in the HTML templates used by the widgets, these should really have been moved into the CSS files, allowing more flexibility. <br /></li></ul><ul><li>Creating your own widgets. based on the dojo ones, It looks like a theroetically simple task, but an example in the demo's would probably help out alot (eg. a single file with an extended dojo widget example) - not a sexy demo, but would be really handy..</li></ul><ul><li>Documentation of attributes. as I mentioned before, the two set's of documentation - API and Manual miss out on the esential part of dojo, the use of HTML attributes to change settings. This could probably be generated or displayed in the API manual but is missing.</li></ul><ul><li> Missing widgets, or features lacking. There are many little things missing from dojo, that seem obvious, yet are not there (yet), </li></ul><ul style="margin-left: 40px;"><li>Dialogs that are more complete, the basic dialog is very baren, having no OK/Cancel built in, and no styling that makes it look particularly sexy.</li><li>No Basic Dialog set's like prompts, warnings etc. that just display messages or get simple user feedback</li><li>Interaction of the button widget with things like wizards and dialogs - you would expect these to be used by default on the standard wizard, but the default is a rather poor looking button</li><li>The great love of blue - the standard button widget uses rather a inelegant blue colour, a faded grey would probably be more neutral for generic applications.<br /></li></ul><br />So far though I've been pretty impressed with dojo, so I would recommend it as a good solution for a adding a stylish interferace to rather dull get/post web sites.<br /><br /> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/132/Javascript_Toolkits.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/272/Javascript.html">Javascript</A> | <A href="/blog.php/View/132/Javascript Toolkits.html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">25 Feb 2007</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/131/Walter_showing_off_Digitalmars_D_cool_features.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">Walter showing off Digitalmars D cool features</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"> Normally I dont just post a link, but this one is on the 'have to see' list. Walter Bright, the author of Digitalmars D, gave a talk to a C++ group about some of the cool features of D. - reserve an hour to listen to this one.<br /><br /><a href="http://video.google.com/videoplay?docid=-7073020265668105471">Video of Walter Bright at the C++ group.</a><br /><a href="http://www.nwcpp.org/Downloads/2007/AdvancedD/index.html">Slides of Walter Brights talk at the C++ group</a><br /><a href="http://www.nwcpp.org/Meetings/2007/01.html">Event details</a> for reference<br /><br />Some of the stuff like scope(exit) - makes Exceptions actually usable. Some of the other stuff like templates I'm still trying to get my head round, let alone decide how the fit into writing maintainable code....<br /><br /><br /> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/131/Walter_showing_off_Digitalmars_D_cool_features.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/271/Digitalmars D.html">Digitalmars D</A> | <A href="/blog.php/View/131/Walter showing off Digitalmars D cool features.html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">31 Jan 2007</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/130/Backtracing_segfaults_in_a_daemon_in_digitalmars_D.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">Backtracing segfaults in a daemon in digitalmars D</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"> One the projects I'm working on is a SyncML server, written from scratch in D, It's currently in testing mode, and we found that the server was mysteriously crashing. Unfortunatly, since it's threaded, and forked as a daemon, we didn't really want to run it under GDB, (and since GDB segfaults on startup anyway). we where at a bit of a quagmire about how to find the bug.<br /><br />So after a bit of searching through code.google.com I came across the idea of catching the SIGSEGV signal and calling <a href="http://www.gnu.org/software/libc/manual/html_node/Backtraces.html">backtrace</a> and backtrace_symbols <br /><br />This little trick can product output that looks something like<br /><pre>/path/to/application [0xAAAAAA] << address of code<br />/path/to/application [0xAAAAAA] << address of code<br />/path/to/application [0xAAAAAA] << address of code<br />/path/to/application [0xAAAAAA] << address of code</pre>which initially seemed a bit cryptic, but by putting it together with <br /><a href="http://www.gnu.org/software/binutils/manual/html_chapter/binutils_10.html">addr2line</a> can result in some great debugging information.<span style="font-family: monospace;"><br /><br /></span>This is my little backtrace logger for the deamon logger.<br /><pre>static void print_trace()<br />{<br /> <br /> <br /> void *btarray[10];<br /> size_t size;<br /> char **strings;<br /> size_t i;<br /> pid_t pid = getpid();<br /><br /> //writefln("SEG");<br /> size = backtrace(cast(void**)btarray, 10);<br /> strings = backtrace_symbols(cast(void**)btarray, size);<br /><br /> std.process.system("/bin/echo '----BACKTRACE------' " ~ <br /> "> /var/log/myproject/backtrace.log");<br /> <br /> for(i = 0; i < size; i++) {<br /> <br /> char[] line = std.string.toString(strings[i]);<br /> char[][] bits = std.string.split(line, "[");<br /> char[] left = std.string.strip(bits[0]);<br /> if (!left.length) {<br /> continue;<br /> }<br /> // skip lines with ( in them...<br /> if (std.string.find(left,"(") > -1) {<br /> continue;<br /> }<br /> <br /> char[] addr = bits[1][2..length-1];<br /> <br /> <br /> std.process.system("/bin/echo '----" ~ addr <br /> ~ "------' >> /var/log/myproject/backtrace.log");<br /> std.process.system("/usr/bin/addr2line -f -e " ~ <br /> left ~ " " ~ addr ~ " >> /var/log/myproject/myproject.log");<br /> <br /> <br /> }<br /> free(strings);<br />}<br /><br /></pre>of course you need to use a few C externs to make this work:<br /><pre>extern (C) {<br /> int backtrace(void **__array, int __size);<br /> char** backtrace_symbols(void **__array, int __size);<br /> pid_t getpid();<br /> sighandler_t signal(int signum, sighandler_t handler);<br /> void sigsegv(int sig)<br /> {<br /> // reset the handler.<br /> signal(SIGSEGV, cast(sighandler_t) 0);<br /> print_trace();<br /> // really die<br /> exit(SIGSEGV);<br /> }<br /> <br />}<br /><br /></pre>and to add it to you application, stick this in main() somewhere<br /><pre>signal(SIGSEGV, &sigsegv);<br /></pre>testing it is quite simple, just do this in D<br /><br /><pre>void testSegfault()<br />{<br /> <br /> class SegTest {<br /> void test() {}<br /> }<br /> SegTest a;<br /> a.test();<br />}<br /></pre>Now looking at the debug file, you can work out where it failed...<br /><pre>----BACKTRACE------<br />----805e971------ <span style="font-style: italic;">(THIS IS MY OUTPUT CODE)</span><br />_D9myproject7cmdLine6daemon11print_traceFZv<br />init.c/src/myproject/cmdLine/daemon.d:306<br />----805e46b------ <span style="font-style: italic;"><br /></span>sigsegv<br />init.c/src/myproject/cmdLine/daemon.d:121<br />----804db18------ <span style="font-style: italic;">(AND NOW FOR THE LOCATION OF THE SEGFAULT)</span><span style="font-style: italic;"></span><br />_D9myprojectfort7manager7manager7runOptsFZAa<br />init.c/src/myproject/manager.d:50<br />----805617a------<br />_D9myproject10webRequest10webRequest5parseFAaAaKAaZAa<br />init.c/src/myproject/webRequest.d:89<br />----8050c3e------<br />_D9pmyproject14myprojectThread14myprojectThread18dealWithWebRequestFAaAaZv<br />init.c/src/myproject/myprojectThread.d:331<br />----80503d0------<br />_D9myproject14myprojectThread14myprojectThread3runFZi<br />init.c/src/myproject/myprojectThread.d:111<br />----8076260------<br />_D3std6thread6Thread11threadstartUPvZPv<br />??:0<br />----a7fd10bd------<br />??<br /><br /></pre>I'm sure with some more work, you could get it to log to syslog...<br /> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/130/Backtracing_segfaults_in_a_daemon_in_digitalmars_D.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/271/Digitalmars D.html">Digitalmars D</A> | <A href="/blog.php/View/130/Backtracing segfaults in a daemon in digitalmars D.html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">24 Jan 2007</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/129/Wing_Lung_Bank_Hong_Kong_not_exactly_trustable.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">Wing Lung Bank (Hong Kong) not exactly trustable.</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"><p>Banks, while never perfect corporate citizens usually at least try to behave in a reasonable maner. That is probably why I was fuming with anger at Wing Lung Bank in Hong Kong.<br /><br />I have a credit card from this bank, that was given out as HKIA uses them for their membership cards, and my spouse being a member, just signed me up for it. I normally use American Express for most payments, but some shops don't accept it so having a Visa card is quite handy.</p><p>We always pay off the balance every month, I normally sit down around the 1st or 2nd of each month and clear the bills. I must have been doing this for at least 6 years, and I've had the Wing Lung card for I guess more than 3 years.</p><p>So yesterday when I got a letter from them indicateing we had not paid the bill, I was a little supprised. That turned out to be me forgetting to pay my wife's bill last month, as she doesn't use the card normally, so the is normally nothing to pay. However while checking the other bills, I noticed a finance charge, and a late payments charge. I was pretty certain that I had paid last month on time, so I checked closer, I had paid the bill on the second of the month, which was the usual date. Then I spotted the problem..<br /><br /><span style="font-weight: bold;">WING LUNG, THE BUNCH OF CROOKS HAD CHANGED THE PAYMENT DUE DATE!!!!</span><br /><br />They had moved it back about 10 days, resulting in the card running up late charges.<br /><br />Not only that, I started checking the older statements,and realized the had pulled this slight of hand 5 months ago, raking in a tidy sum for their criminal coffers.<br /><br />By this time I was fuming, and called their hotline, my temper not subdued by their annoying 'press 2 for..' and playing adverts for their services while waiting.<br /><br />After practicalling screaming at two idiots who refused to refund the money and change the payment date, and listening to them try and justify this crazy action 'your agreement states we can change the terms at any time' They promised that a manager would call back.<br /><br />Of- course, this call never arrived, so today I cancelled the card, filed a complaint with our consumer council. (Hopefully at least they will issue a warning about them Changing the due dates, so other customers don't get caught out.)</p><p>From what I've been hearing, this is not the first bank to pull this scam.. Well, Wing Lung will not be one of the place's I'm going to get a mortgage from...<br /></p> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/129/Wing_Lung_Bank_Hong_Kong_not_exactly_trustable.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/267/Hong Kong.html">Hong Kong</A> | <A href="/blog.php/View/129/Wing Lung Bank (Hong Kong) not exactly trustable..html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">06 Jan 2007</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/128/Hong_Kong_consultation_on_copyright_changes.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">Hong Kong consultation on copyright changes.</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"> <!-- ======================================================= --><!-- Created by AbiWord, a free, Open Source wordprocessor. --><!-- For more information visit http://www.abisource.com. --><!-- ======================================================= --> <div> <p>Response to <a href="http://www.citb.gov.hk/cib/ehtml/pdf/consultation/Consultation_document.pdf">Copyright Changes consultation document</a>:</p> <p> </p><p>Please Have a look through my response, (I'm not an qualified activist, or an English major - so you probably want to improve it) but please do respond if you live in Hong Kong. It's already a bad enough police state, without these idiots letting this kind of legislation loose.</p> <p> </p><p> </p><p> </p><p> </p><p> </p><p>To co_review@citb.gov.hk</p> <p> </p><p style="font-weight: bold;">Introduction:</p> <p> </p><p>The whole premise of this document is outrageous, it reads like a music industry wish-list, and follows on from a previous consultancy document that was rejected before. Basically trying to force a similar attitude without care or compassion for citizens of Hong Kong.</p><p> <!-- ======================================================= --><!-- Created by AbiWord, a free, Open Source wordprocessor. --><!-- For more information visit http://www.abisource.com. --><!-- ======================================================= --> </p><div> <p>It Completely ignores fair use and consumer rights issues, which have already been seriously damaged by the current legislation, and are railroaded in favor of helping the antiquated business models of copyright monopolies. A particularly bad example of collusion between government and business over consumer rights.</p> </div> <p> </p><p> </p><p><!-- ======================================================= --><!-- Created by AbiWord, a free, Open Source wordprocessor. --><!-- For more information visit http://www.abisource.com. --><!-- ======================================================= --></p><div><p><br /></p> </div> <p> </p></div> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/128/Hong_Kong_consultation_on_copyright_changes.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/267/Hong Kong.html">Hong Kong</A> | <A href="/blog.php/View/128/Hong Kong consultation on copyright changes..html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">22 Dec 2006</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/127/local_caching_and_JSON_bugs.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">local caching and JSON bugs.</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"> One of the projects I'm working on is using XUL to render data, we started testing a few weeks ago, and discovered that the load times/ response where quite bad. It was pulling too much data from the server, and the application would either hang (if we didn't use asyncrounous xmlhttprequest calls), or we would have to show a slidy to indicate that data was being loaded. Neither where ideal.<br /><br />In discussing the application with the client, they also requested that the application remember some of the states (eg. the data filters). Normally you might consider cookies or having a preferences database, but I remember seeing something about firefox2 having localized session storage.<br /><br />You can read the <a href="http://www.whatwg.org/specs/web-apps/current-work/#storage">session storage specification here</a>, It allows you to store strings totalling upto about 5Mb on the client side. However, the reality is that you really need to be able to store javascript objects or similar in it. So since we where using JSON to transfer data, I thought 'how about serializing the data back into JSON, and storing it in the session.'<br /><br />I started off by using the <a href="http://www.json.org/json.js">JSON js</a> code, <span style="font-weight: bold;">however this code is unusable for 1 major reason</span> - it adds a method to array's/objects, resulting in the fact that foreach() always returns an additional member. (breaking almost all the existing javascript code!) - I also ran into problems with recursion in firefox (but I think thats due to me forgetting to add the String.prototype method.)<br /><br />So in the extended entry is a quick kludge (without full unicode support) for converting items into JSON in javascript, so that something like this works:<br /><br /><pre>sessionStorage.setItem("mycache_somedata", JSON_toString(mydata));</pre><p>... and onloading..</p><pre>var mydata = new Array();<br />if (sessionStorage.getItem("mycache_somedata")) {<br /> eval("mydata = " + sessionStorage.getItem("mycache_somedata").value);<br />}<br /></pre> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/127/local_caching_and_JSON_bugs.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/270/XUL.html">XUL</A> | <A href="/blog.php/View/127/local caching and JSON bugs..html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">12 Dec 2006</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/126/Autocompletion_in_leds_for_Digitalmars_D.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">Autocompletion in leds for Digitalmars D</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"> <p>I've been spending far to much time looking at the autocompletion in <a href="http://www.dsource.org/projects/leds">leds</a> for D. The justification, while growing weaker the more time i spend on it, remains the same. Autocompletion and AutoHelp save a huge amount of time when writing code.<br /><br />The PHP parser in leds is working quite well, is organically written, and has room to grow, the next major step is guessing object types by matching method calls on unknown objects, so while I spend about the same amount of time coding in D as I do in PHP now, I had decided to look at the D parser, and push that along.<br /><br />Antonio had written the <a href="http://svn.dsource.org/projects/dantfw/trunk/src/parser/">current parser</a> which is in the <a href="http://www.dsource.org/projects/dantfw">dantfw</a> project. It aimed to parse all C like languages, C, D, Java and C# etc. This meant that rather than following the D specification, the tokenizer and parser where quite generic. and lead to two issues.<br /></p><ul><li>When I added autocompletion as you type, rather than on demand (ctrl space), I noticed that it was attempting to autocomplete when i was typing "strings" and comments. </li><li>Quite often it appeared to be missing some key variables or methods that it should have known about.</li></ul><p><br />The first issue I resolved by rewriteing the tokenizer (following the D specification closely), and re-tokenizing the document as you type (if it thinks your scope may have changed), then deducing the scope being input.<br /><br />The second, I began to conclude was more related to the generic nature of the parser. It was making many assuptions about the language that turned out to be incorrect in D's context. The only, rather drastic solution was to rewrite the parser....<br /><br />So on rather a quiet day i started that thankless task. Using Antonio's original design, of using object contructors to parse and eat their own tokens, I set about writing loads of switch/case combos.</p><p>My first effort worked reasonably well, but the more i wrote it, I began to realize that both the complexity, and the use of a series of tokens had a number of flaws. The declaration code for parsing methods, and variables was quite large, and very similar. (ending in a lot of redundant, duplicate code) Declaring multiple variables in one line also made the resulting abstract syntax tree objects quite klunky.<br /><br />At this point I put it on hold for a while, partly to consider alternative approaches, and more to ensure paid work was not falling behind.<br /><br />When i returned, I had come up with a few ideas<br /></p><ul><li>look at dmd for inspiration </li><li>check dsource to see if somebody had already done this</li><li>consider preparsing tokens into a tree before sending them to the parser.</li></ul>From studing the dmd parser, I realized that Walter had gone for a parser Object in C++, (rather than our constructor parsing) and broken the different scopes into different parsing routines When the current parsing routine they hit a syntax patern that matched it called the parse do deal with that scope, that in turn added a new object to the AST Stack. He also appeared to skip all Comments/Whitespace and EOL's from the token fetching routine, (actually merging the comment block into the next token, although it's a touch more complex than that.)<br /><br />From dsource I found codeanalyser, which after a bit of hacking to remove what appeared to be the C++ memory stack allocation routines. I finally got to compile (without crashing dmd), parse and output what looked a little bit like syntax trees. <br /><br />The downside to this project was that extracting the detailed information I required (type definitions, method declarations, line numbers etc) was not really feasible, and the tree that was produced by the code included a significant amount of noise, (in that it frequently created tree nodes for failed syntax matches). Along with this, it would require some work to store the token data within the tree that was created.<br /><br />On the upside, in debugging the compiling issues I began to get a better  understanding Templates (one of those black magic features of D). I ponder if the introduction to templates in D should basically start with the statement.<br />"D does not have preprocessor macro's, it has Templates" as while Templates are considerably more powerfull than Macro's,  from a comprehension point of view, that is effectively what they are. And, given that  D's decision not to have macro's, on what appears to be a sensible view that they obfusicate the code (along with often making it a nightmare to compile),  I consider Templates to be on the list of features to be used with extreme caution, as they have a similar obfusicating effect, although perhaps not to the same degree....<br /><br />Anyway, so the last option was to chuck my first draft, and use an idea of building a token tree, and passing that to the parser, rather than just giving a series of tokens. Having got about half as far as I did before, I consider this one of those little gem's of an idea. That makes writing the Parser considerably simpler..<br /><br />Consider the simple statement<br /><pre>void main(char[][] argv) { }</pre>Which would tokenize into the following.<br /><pre>Token.T_void : void<br />Token.IDENTIFIER main<br />'('<br />Token.T_char char;<br />'['<br />']'<br />'['<br />']'<br />Token.IDENTIFIER argv<br />')'<br />'{'<br />'}'<br /></pre> <br />Using a post tokenizing tree building routine, it now looks like this, which from the perspective of the parser, is considerably simpler to deal with.<br /><br /><pre>Token.T_void : void<br />Token.IDENTIFIER main<br />'('<br /> Token.T_char char;<br /> '['<br /> ']'<br /> '['<br /> ']'<br /> Token.IDENTIFIER argv<br /> ')'<br />'{'<br /> '}'<br /></pre> From a pattern perspective, we are just looking for <br /><pre>BaseType, IDENTIFIER '(' '{' == a method declaration.</pre>(although it's a bit more complex than that in real life!)<br /><br />The tokens inside of a '(', '{', '[' collapse into that token, so we can either ignore them, when not needed, or send them as a set to a parsing routine. which doesnt have to keep dealing with nesting or determining the closer.<br /><br />Anyway, the new parser is slowly ticking away, once it parses correctly, the next step is to work out how to make the resolver cleaner...<br /> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/126/Autocompletion_in_leds_for_Digitalmars_D.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/271/Digitalmars D.html">Digitalmars D</A> | <A href="/blog.php/View/126/Autocompletion in leds for Digitalmars D.html#comments">Add / View Comments()</A> </DIV> </DIV><DIV class=" well" xbuilderid="roo-gen45" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-gen46" xtype="Roo.bootstrap.Header">24 Nov 2006</H3> <H2 xbuilderid="roo-gen47" xtype="Roo.bootstrap.Header"> <A href="/blog.php/View/124/Multilanguage_setup_for_Flexy_and_FlexyFramework.html" xbuilderid="roo-gen48" xtype="Roo.bootstrap.Link">Multilanguage setup for Flexy and FlexyFramework.</A> </H2> <DIV class="" xbuilderid="roo-gen49" xtype="Roo.bootstrap.Element"> Having just released the increadibly minor update to the Flexy Template Engine in pear, I can now release the documentation that goes with it... <br /><br />FlexyFramework, with HTML_Template_Flexy include the ability quickly create multilingual websites, by translating the templates.<br /><br />I've set this up a few times now, and kept meaning to document it. - So here it goes.<br /><br />Having set up FlexyFramework, (see the previous post about this), <br /><br />... The full instructions are in the Extended entry....<br /><br /> </DIV> <DIV class="" xbuilderid="roo-gen50" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/124/Multilanguage_setup_for_Flexy_and_FlexyFramework.html" xbuilderid="roo-gen51" xtype="Roo.bootstrap.Link">View Extended Entry</A> </DIV> <DIV class="" xbuilderid="roo-gen52" xtype="Roo.bootstrap.Element">Posted by in <A href="/blog.php/BlogCategory/266/PEAR.html">PEAR</A> | <A href="/blog.php/View/124/Multilanguage setup for Flexy and FlexyFramework..html#comments">Add / View Comments()</A> </DIV> </DIV> <DIV class=" col-md-8 maincontent" xbuilderid="roo-gen54" xtype="Roo.bootstrap.Container"> <DIV class="text-center text-center" xbuilderid="roo-gen55" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View.html?page=10" xbuilderid="roo-gen56" xtype="Roo.bootstrap.Link">« prev page</A> <SPAN class="" xbuilderid="roo-gen57" xtype="Roo.bootstrap.Element">   (Page 11 of 24, totalling 233 entries)   </SPAN> <A href="/blog.php/View.html?page=12" xbuilderid="roo-gen58" xtype="Roo.bootstrap.Link">next page »</A> </DIV> </DIV> </DIV> </DIV> </DIV> </SECTION> <DIV xbuilderid="roo-comp-1003" xtype="Roo.bootstrap.Element"> <SECTION class=" social" xbuilderid="roo-comp-1005" xtype="Roo.bootstrap.Container"> <H3 xbuilderid="roo-comp-1007" xtype="Roo.bootstrap.Header"> Follow us on <A href="http://www.roojs.com/index.php/BlogRSS/0/All%20Categories.xml"> <IMG src="/Roojscom/templates/images/rss_32.png" width="32" height="32"></IMG> </A> <A href="http://www.linkedin.com/in/roojs"> <IMG src="/Roojscom/templates/images/linkedin_32.png" width="32" height="32"></IMG> </A> <A href="http://twitter.com/roojs"> <IMG src="/Roojscom/templates/images/twitter_32.png" width="32" height="32"></IMG> </A> </H3> <DIV class="row clearfix" xbuilderid="roo-comp-1009" xtype="Roo.bootstrap.Row"> <DIV class="column col-md-6" xbuilderid="roo-comp-1011" xtype="Roo.bootstrap.Column"> <DIV class=" alert alert-warning" xbuilderid="roo-comp-1013" xtype="Roo.bootstrap.Container"> <H2 xbuilderid="roo-comp-1015" class="head-set" xtype="Roo.bootstrap.Header">OUR BLOG</H2> <DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/304/Some thoughts on the language server and its usefulness in the roobuilder " xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">Some thoughts on the language server and its usefulness in the roobuilder </A> </DIV><DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/303/Roo Builder for Gtk4 moving forward" xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">Roo Builder for Gtk4 moving forward</A> </DIV><DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/297/Clustered Web Applications - Mysql and File replication" xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">Clustered Web Applications - Mysql and File replication</A> </DIV><DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/293/GitLive - Branching - Merging" xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">GitLive - Branching - Merging</A> </DIV><DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/292/PDO_DataObject Released" xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">PDO_DataObject Released</A> </DIV><DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/291/PDO_DataObject is under way" xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">PDO_DataObject is under way</A> </DIV><DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/288/Mass email Marketing and anti-spam - some of the how-to.." xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">Mass email Marketing and anti-spam - some of the how-to..</A> </DIV><DIV xbuilderid="roo-comp-1017" xtype="Roo.bootstrap.Element"> <A href="/blog.php/View/287/Hydra - Recruitment done right" xbuilderid="roo-comp-1019" xtype="Roo.bootstrap.Link">Hydra - Recruitment done right</A> </DIV> </DIV> </DIV> <DIV class="column col-md-6" xbuilderid="roo-comp-1021" xtype="Roo.bootstrap.Column"> <DIV class=" alert alert-warning" xbuilderid="roo-comp-1023" xtype="Roo.bootstrap.Container"> <DIV xbuilderid="roo-comp-1025" xtype="Roo.bootstrap.Element"> <A class="twitter-timeline" data-dnt="true" href="https://twitter.com/roojs" data-widget-id="478410483276328960"> </A> </DIV> </DIV> </DIV> </DIV> </SECTION> <A name="contact" xbuilderid="roo-comp-1027" style="position:relative; top:-30px" xtype="Roo.bootstrap.Link"></A> <FOOTER class=" top-space" xbuilderid="roo-comp-1029" xtype="Roo.bootstrap.Container"> <DIV class="container" xbuilderid="roo-comp-1031" xtype="Roo.bootstrap.Container"> <DIV class="container" xbuilderid="roo-comp-1033" xtype="Roo.bootstrap.Container"> <DIV class="row clearfix" xbuilderid="roo-comp-1035" xtype="Roo.bootstrap.Row"> <DIV class="column col-md-3 widget" xbuilderid="roo-comp-1037" xtype="Roo.bootstrap.Column"> <H3 xbuilderid="roo-comp-1039" class="widget-title" xtype="Roo.bootstrap.Header">Contact</H3> <DIV xbuilderid="roo-comp-1041" class="widget-body" xtype="Roo.bootstrap.Element"> <P xbuilderid="roo-comp-1043" xtype="Roo.bootstrap.Element"> <BR /> Email: <A href="mailto:sales@roojs.com">sales@roojs.com</A> <BR /> Tel: <A href="Tel:+85237092951">+852 3709 2951</A> <BR /> Room 2710, Trend Center <BR /> 29-31 Cheung Lee Street, <BR /> Chai Wan <BR /> Hong Kong <BR /> </P> </DIV> </DIV> <DIV class="column col-md-offset-4 col-md-4 widget" xbuilderid="roo-comp-1045" xtype="Roo.bootstrap.Column"> <DIV xbuilderid="roo-comp-1047" class="widget-body" xtype="Roo.bootstrap.Element"> <P xbuilderid="roo-comp-1049" xtype="Roo.bootstrap.Element">Copyright 2023 - Roo J Solutions Limited</P> <P xbuilderid="roo-comp-1051" xtype="Roo.bootstrap.Element">Powered by the Roo Javascript Library and our Roo bootstrap toolkit</P> </DIV> </DIV> </DIV> </DIV> </DIV> </FOOTER> </DIV> </body> <script type="text/javascript" src="/roojs1/roojs-core-debug.js?ts=1234"></script> <script type="text/javascript" src="/roojs1/roojs-bootstrap-debug.js?ts=1234"></script> <script type='text/javascript'> var baseURL="\/blog.php"; var rootURL=""; var authenticated=null; var error_data=null; var page_args=null; </script> <script type="text/javascript" src="/Roojscom/Roojscom.js"></script> <script type="text/javascript" src="/Roojscom/Header.js"></script> <script type="text/javascript" src="/Roojscom/Footer.js"></script> <script type="text/javascript" src="/Roojscom/ContactDialog.js"></script> <script type="text/javascript" src="/Roojscom/ViewArticles.js"></script> <script type="text/javascript"> Roojscom.init(); </script> </script> <script type="application/ld+json"> { "@context": "http://schema.org", "@type": "Organization", "name" : "Roo J Solutions Ltd.", "url": "http://www.roojs.com", "sameAs" : [ "https://www.facebook.com/roojs", "https://www.linkedin.com/company/roo-j-solutions-ltd-", "https://twitter.com/roojs" ] } </script> <script type="application/ld+json"> { "@context": "http://schema.org", "@type": "LocalBusiness", "name" : "Roo J Solutions Ltd.", "url": "http://www.roojs.com", "logo": "https://www.roojs.com/Roojscom/templates/images/roojsolutions-tr-100.png", "description": "Hong Kong's Premier SME Software Development Service", "telephone": "+852 3709 2951 ", "address": { "@type": "PostalAddress", "addressLocality": "Hong Kong", "addressRegion": "HK", "streetAddress": "Room 2710, Trend Center, 29-31 Cheung Lee Street" }, "openingHours": [ "Mo-Fr 09:00-17:00" ] } </script> </html>