Pythoneer

… we will be able to deliver much quicker, much cooler applications to the user. No installation needed, just an online connection. Not that I was not already talking about that for a while, but there was no hook between the position that your device knows and the web app you are running. Google Geear LocationAPI will provide it. It’s great to see the technologies go the right way to really provide the power to enable easier writing of really good apps. Let’s see what people come up with …
Btw, watch this video it is very nice in showing what gears is good for and what is yet to come, and you can hear dojo over and over again :-)!

dojo.explore("Berlin", "Saturday May 31, 2008 at 3pm");
dojo.beer();
dojo.dinner();
notTooTired = true;
while (notTooTired) {
    dojo.clubbing();
}

Stay updated via the upcoming event page.

There is a ton of helpful stuff in dojo and it is getting a lot better and more. Even the documentation is becoming better, so don’t hesitate to try it out! And there are some little pieces that you see coming every day that are being added to dojo to help.

There is for one the daily (if not hourly) progressing API viewer.
And there is the new dojox.help stuff.
And as a tiny little bit I also have added a piece (the apropos command) that probably gets added into the dojox.help namespace.

Stay on the pulse of dojo!

Now that I dove into functional programming (only using dojo and python, but it’s a start) and feel that my brain has not completely made the switch yet, I come across problems that look like they are best to be solved with functional programming. Isn’t it always that way, when you have started something new and are still excited about it, that the every day problems seem that they want to be solved the new way? Anyway, here it goes.

I got this dictionary. The keys are IDs and the values are kind of a rank. Now I want to sort the IDs by their rank.
I am sure there is a simple one liner for that. But I didn’t come up with it yet.
I only came up with a quite dumb concat, sort and split solution. Not elegant at all.

d = {4: 6, 12: 4, 14: 14, 18: 16, 32: 6,
    33: 24, 40: 27, 41: 6, 44: 25, 106: 2}
d1 = ["%06d" % v+'_'+str(k) for k,v in d.items()]
# d1 is now something like this:
# ['000006_4', '000004_12', '000014_14', '000016_18', ...]
d1.sort() # That allows us to sort, which does it
sorted = [int(i.split("_")[1]) for i in d1 ]

This has several drawbacks, but it was as good as I got :-(.
I hope this has no negative effect on my career.

[Update] Thanks to Daniel’s second post, this is the solution!

sorted(d.items(), key=itemgetter(1))

Awesome, thanks!

After reporting the mentioned bug to the Firefox guys they were not able to confirm it, and they are right!
It seems to be a Firebug bug!
Because if I do the same tests inline in the Firebug Lite console (not the Firefox extension in the browser!) everything works as expected.

>>> [] instanceof Array
true

I keep investigating

See Alex’s blog entry he posted the problem with Firefox3 beta3!
In short

>>> typeof []
"object"
>>> var a = [];
>>> var b = new Array();
>>> a.constructor == b.constructor
false
>>> c = [];
[]
>>> c.constructor == a.constructor
true
>>> d = new Array();
[]
>>> b.constructor == d.constructor;
true
>>> b.constructor == a.constructor;
false

After reading Eugene’s article about functional fun with dojo I wanted to play with dojox.lang.functional, and got stuck right away on one of the first calls.

>>> dojo.require("dojox.lang.functional")
>>> dojo.require("dojox.lang.functional.fold")
>>> df = dojox.lang.functional
>>> df.reduce([1,2,3], "+")
TypeError: a.hasNext is not a function

Oops, that was not what I expected, thanks to Alex’s help Firefox3 beta3 could be identified as the one who causes the problem. And I filed a bug there, so lets hope the next Firefox beta/version does it right.

[Update] The bug seems to be in FireBug, not in Firefox

Yesterday another one of my contributions found it’s way into dojo: dojox.widget.Rating. You can try out the always latest version online in the dojo archive. It’s quite a simple form widget, that allows you to click a star for rating an item. The number of stars and the styling can be adjusted via images and CSS.
If you want an even more flexible way to customize the stars you should look at what Tobi has to offer with his SVG generated stars, they come in many more flavors!

Just because I just came across it while optimizing some old code, I have to mention this.
I have the following list of dictionaries (array or hash-arrays in other words) and I want to sum up the values of one of the items of the dictionary.

In python I do this

    d = [{'value':1, 'title':'one'}, {'value':2}, {'value':3},
         {'value':4, 'title':'four'}, ]
    s = sum([i['value'] for i in d])

In PHP I would have done this (I guess)

    function extract_value($r) {
        return $r['value'];
    }
    $d = array(array('value'=>1, 'title'=>'one'), array('value'=>2),
               array('value'=>3), array('value'=>4, 'title'=>'four'));
    $s = array_sum(array_map("extract_value", $d));

Cracks come forward and shorten it please (in any language!).

I know that already for a long time. And since using dojo I know it even better. And already for a couple of years I am always again coming across functional programming, but never got any deeper into it then using the basic things. But now there is a really juicy piece of fruit I will chew, thanks to Eugene Lazutkin who wrote this wonderful article on Functional JavaScript fun in Dojo. And he also describes what which JS version implements, and more important which browser provides what. Must read!
Thanks …

Ok, watch out for dojo accelerating and passing on the left … Shane O’Sullivan posted, already in december last year about the dojo demo engine. A great step forward and a way to document stuff much better, since you get examples and one tab further you see how it is coded.

It is still in development, but the future looks bright!
So, use all the resources you get …

Mayflower had invited me to join their Barcamp last weekend - and it was awesome. Thank you a lot!!!
We had been a crowd of about 40 people all hacking all day long until deep at night, it was a great atmosphere and some very interesting projects had been made. I am sure some things will come up on the Mayflower blog soon, just keep watching!
I was mainly working on integrating the dojo.grid into the next version of Phprojekt and doing some ground work for a very flexible structure. And I have learned a lot more about dojo again. It’s just great!
Besides that I was advocating a lot for Python and Django especially :-), I just can’t hold back. But it’s hard to convert anyone there, since Mayflower is the biggest and best PHP service provider in Germany and the people there are really good in what they are doing! Though one came up to me and said “thank you for putting the Python book on the table” (which I did a while ago in the Mayflower office, just for fun).
Thanks again and hopefully see you soon again.

From time to time I got this UnicodeEncodeError, but I had done all the things (sitecustomize.py, some more) right in order to configure the system in UTF-8. I thought.

I did the usual

     ./manage runserver

and since I assumed that on my Mac everything is already properly configured to be UTF-8 I didn’t even think of the shell not being properly set up. Fortunately my colleague Tobi found out that I have to set the shell variable LANG=”utf-8″. So now I always do:

     export LANG="utf-8"
     ./manage runserver

So I went from the wrong locale output to the one working properly with UTF-8 and Django, jiha!

cain@home:~> locale
LANG=
...
cain@home:~> export LANG="utf-8"
cain@home:~> locale
LANG="utf-8"
...
cain@home:~> # now i can safely start Django 
cain@home:~> ./manage runserver

[UPDATE] The link to the sourcecode now points to the dojango implementation, the dpaste one is gone.

Just in case someone is using json_encode() I wrote a while ago, I have updated the function due to updates in the latest Django dev version. Find the source here.

I have solved two problems here. One is that now lazy strings are used all over in Django, which this updates handles and detecting a list and a dictionary stumbled on subclasses of them.

I just changed the config of my application and tested it if it still works, since I only switched DB names. Everything is cool. It works, all the pages show the same stuff it seems to work. Great thing!
A couple minutes later, I go to the shell window where I did that change and am wondering what server I am on. Geeeeeeee, the online version. I changed the DB to use on the live version. God damn it.

It was no big deal, since I hadn’t restarted the server yet, since my local dev server (django) automatically restarts when I change any python file. So I just undid the change online.
Something has to change that this won’t happen again, searching the web a bit and with the help of Daniel I found this article “Colorful Terminal”, where he had solved the same problem. Great solution.
Just one little thing, when I accidently forget to start the script which logs me into the server and changes the background color of my terminal … what then? Can the script not trigger on “ssh me@server.com”? Any ideas?

I just found eye-fi via the Sitepen blog. Just mix wifi and a flash card and you got more spare time and less cables hanging around. Cool.

A short time ago I found this very well done podcast: Hanselminutes by Scott Hanselman. In one of the latest episodes he was talking to Jonathan Zuck of ACT Online about Software Licensing. It’s great to get a quick overview about existing licenses and how to best use them. Since this is a huge topic you only get a short overview, but very understandable imho!
Especially interesting was what Jonathan said about the MySQL licensing model, which I had not properly understood all the way until now neither :-). He said since MySQL owns the rights of the software they can decide what license to sell a software under to a customer and

relief them of the burden of a GPL based license, by simply selling it to them commercially.

(starts at 27:30). That’s pretty well explained I think. It makes all this licensing stuff much simpler in my eyes.
Thanks Scott for your great podcasts!

I guess by now everyone knows that the dojo version 1.0 had been released yesterday. I will closely follow the dojo development, since I think this is the most promising JavaScript framework out there.
Let’s hack on …

Joseph Smarr from Plaxo was holding a talk about High-performance JavaScript: Why Everything You’ve Been Taught Is Wrong. His blog with some background info is here. So many true things and some new things, very interesting to watch if you are interested in tweaking your AJAX app to be performing great.

It was especially interesting to hear one thing that I also found out in my current project, it was “do DOM manipulation off-DOM”. Which means if you are doing some “heavy” DOM manipulation you should ie. copy the DOM element into an absolute positioned element (so that it doesn’t effect all the other elements that are floating around it) and manipulate it there. What I did in this case, was just simply setting visibility to hidden, so I can manipulate and evaluate all the parameters (such as layer size, position, etc. mostly using dojo.corrds()) and when done calculating I simply apply it to the original element. In our case we were resizing fonts to best fit in a box. And it made a noteable difference in performance to make the font size manipulation off-DOM.

Found via Alex’s blog entry on the dojo page, thanks.

First things first, if your project allows you to migrate to 0.9 (you got the time to do it) - do it! It’s worth all the sweat and it might cost you some.

I have converted a mid-size project from dojo 0.4 to 0.9 and it took longer than I thought, but also because I had to collect all the info from multiple places, so I will try to share what I discovered or collected and hoepfully make it easier to migrate. Good luck!
If you come across more changes that match somewhere in the list just add them to the comments, I will try to incorporate them.

Namespaces
Some functions have changed their namespaces. Most common functions have moved to dojo.* namespace to simply be quickly reachable. Simply do a console.dir(dojo) on the FireBug console, which lists all the functions that are located in the dojo.* namespace.
I found those changes:

dojo.lang.* => dojo.*
dojo.lang.find => dojo.lang.indexOf
dojo.lang.filter => dojo.filter

dojo.html.* => dojo.*
dojo.html.hasClass => dojo.hasClass
dojo.html.addClass => dojo.addClass
dojo.html.removeClass => dojo.removeClass
dojo.html.toggleClass => dojo.toggleClass

dojo.widget.byId => dijit.byId
dijit.byNode is new, the speeks for itself imho

Some functions disappeared
dojo.lang.isUndefined() disappeared, I didnt find the reasoning anywhere yet. If you have no deeply nested objects you try to check, a function isUndefined() will work well I created one for my project. If you don’t like that I guess you have to replace all calls with typeof varName == "undefined".

dojo.lang.isNumber() disappeared too. My solution:
function isNumeric(v) {return dojo.number.format(v)!==null};.

dojo.html.getPadding() was removed and seems to have become dojo.style(el, "paddingLeft"), dojo.style(el, "paddingRight"), etc. and that looks reasonable, it only gets the data you really need not all the paddings though you would have needed only one.

dojo.html.getScroll() is gone, the hack I use is dojo._docScroll() which does return the data as I need them, but it looks very private :-(.

dojo.dom.removeNode(el) was removed since the same can easily be achieved el.parentNode.removeChild(el), though removeNode() was pretty convinient, but not really necessary. Another intelligent move.

dojo.string.escape() is gone too. I just copied the old 0.4 function (dojo.string.escapeXml() see here for the source) into my local project namespace, I didnt find a replacement in dojo0.9 yet :-(. Probably also a simple thing like the removeNode thing, but I don’t know yet.

Important function changes
dojo.html.toCoordinateObject() was renamed to dojo.coords() which is a good move, since it was way too long. It now returns an object with the keys: w,h,t,l (width, height, top, left). Much more efficient.

this._inherited() is not a function anymore, but I guess I was one of the few who used it directly though I shouldn’t. Now call this.inherited("funcName", arguments).
I think that only due to a bug or missing implementation I used to use _inherited() now it seems to be fixed and as it is supposed to be.
But: inherited() can only be called for the same function as the callee … strange, so we have to use className.functionName.apply(this, arguments) and that is one thing that really sucks, since you hard code the className in the class … I hope that will change.

dojo.fx.html.wipeOut(el, 1000, null, onEnd) changes to this dojo.fx.wipeOut({node:el, duration:1000, easing:null, onEnd:onEnd}).
For all the properties see the dojo._Animation object’s properties are all you can pass to the wipe-method.

Widget stuff
djConfig.searchIds forget about them!
Alex Russel told me in IRC
[12:35] cain: what is the replacement for “djConfig.searchIds” ?
[12:35] slightlyoff: cain: there isn’t one
[12:35] cain: how do i add ids to be rendered as widgets?
[12:35] slightlyoff: cain: there’s only a larger toggle (parseOnLoad)
[12:35] slightlyoff: cain: or you can run the parser yourself on a smaller subtree
[12:35] slightlyoff: cain: but in 0.9, the parser is much much much faster
So all you need to do is put djConfig.parseOnLoad=true at the beginning of your page and all widgets will be found. But be careful you have to manually dojo.require all the widget classes, iirc it was done automatically in 0.4.
The porting guide says:

you must require the Dojo parser along with any widgets referenced, and enable a page scan using the “parseOnLoad” setting

that sucks, for the parser could easily find out which class/file to load, now i have to handish require things.

In place of dojo.widget.createWidget(), simply use the Javascript “new” constructor. I.e. var dialog = new dijit.Dialog(params).

Replace all


by
. The use of namespaces for creating widgets is straightforward now, in 0.4 it was strange. Great.

When building your own widgets you surely used the following dojo.uri.moduleUri that has become dojo.moduleUrl.

dijit.form.Form.getValues() is currently ignoring all fields inside the form if they are not widgets :-(, in 0.4 it didn’t.

dojo.widget.ComboBox is now dijit.form.ComboBox remotely loaded data are now realized via the dojo.data API. Before you simply specified a dataUrl attribute now that requires a dojo.data-store. And if you have a huge set of data on the server that you actually want to use the ComboBox to filter on, you will need this patch!

Replace dojo.widget.Form not with dijit.form.Form, but don’t make it a widget and use dojo.formToObject() instead, its much more efficient and explicit.

dojo.widget.Editor2 has become dijit.Editor, the toolbar config is much easier now, it used to be in an external file, now it’s as simple as that:

data in here

There is a problem loading it into a hidden layer, see this ticket there is work on the way see this comment.

Other stuff
In 0.4 it was not such a trivial task to make dojo render widgets that were in a chunk of HTML, that was loaded via AJAX. Now parsing a HTML chunk and render widgets in there simply is a call to dojo.parser.parse(node).

For most uses dojo.io.bind() has become dojo.xhrPost()/dojo.xhrGet() and the return data have been extended quite a bit.

If you have a form, that is a dijit.form.Form widget and you have some extra attributes, that the widget doesnt support by default in there, as i.e. “action” they will be kicked out, I used the following patch http://trac.dojotoolkit.org/ticket/4392 to have them added again. But there seems to be changes going on in this area, there is some attributeMap added, but I am not sure if this is the way and if that stays.

In the bootstrap.js i found the following comment:

Refer to ‘dojo.global’ rather than referring to window to ensure your code runs correctly in contexts other than web browsers (eg: Rhino on a server).

That means (and is also practiced in dojo) dont access “window” directly anymore but via “dojo.global” to me that sounds like it might make a lot of sense to simplify testing outside the browser, but I need to see that in action somehow.

Dojo is going the very efficient and elegant way of using one function as setter and getter in many places, the setter looks like this:
dojo.cookie("name", value, {param:1, ... });
or the getter, which is the same function(!) is simple as that
var value = dojo.cookie("name");
This makes the API so much simpler, the code cleaner and simply simpler. Very handy.

That’s just a plain list of what I have found and remembered to write down. hth

Sometimes when you want to do something right and you are lacking the tools you write them (but very often I don’t ), but in the case of doctests for dojo I saw the need. This time I took the time to write a doctest-class for dojo. It is currently only a ticket in the dojo trac, but some feedback looks pretty promising.

I saw the Ian Bicking had already once written a doctest module, but after skimming over it and thinking about the task, I decided to write it from scratch and a pure dojo fit, since it surely is not much work (as it also turned out, which is a rare case).
I knew the doctests module from the Python, where it obviously also comes from. In the beginning I found doctests ugly, those huge chunks of console-pastes inside my docstring. But after getting to know the value of the doctests by learning django’s newforms completely by a huge doctest I felt what they can do and that they can really help a lot.
Doctests just make you understand the code in the place it is written. If you (as I do) often look into the sources it can be really helpful to see a programming example right there inside the docstring of the class/method/function.

To make it short, I implemented this for JavaScript and it works nicely. I am actually developing a dojo.data-store but in order to do that I needed doctests, that’s where the need came from.
One of dojo’s base functions might look like this now:

dojo.trim = function(/*String*/ str){
	// summary: trims whitespaces from both sides of the string
	// description:
	//	This version of trim() was selected for inclusion into the base
	//	due to its compact size and relatively good performance (see Steven Levithan's blog:
	//	http://blog.stevenlevithan.com/archives/faster-trim-javascript).
	//	The fastest but longest version of this function is going to be placed in dojo.string.
	// examples:
	//	>>> dojo.trim(" trim me ")
	//	"trim me"
	//	>>> dojo.trim("\t\ttrim me ")
	//	"trim me"
	//	>>> dojo.trim(" \t\t trim me \t ")
	//	"trim me"
	//	>>> dojo.trim("trim me")
	//	"trim me"
	//	>>> dojo.trim(" trim me")
	//	"trim me"
	//	>>> dojo.trim("trim me ")
	//	"trim me"
	//	>>> dojo.trim("")
	//	""
	return str.replace(/^\s\s*/, '').replace(/\s\s*$/, '');	// String
}

You can see more examples here and here (search for the doctest starting with “>>>” and you will find some).
Below “examples” you can see the inline doctest. I really just wrote those tests in the FireBug console and copied them later into the code.

Run doctests
The trac ticket in dojo including the files and some info you can find here. You can download the patch right here.
First apply the patch, so that the dojox.testing.DocTest module is available.
This patch containing the doctest class is though to be a part of dojo, so it currently requires that you have a dojo module or a module that you registered with dojo (dojo.registerModulePath()).
To run i.e. the above shown doctest you just need to know in which file (module) it is (dojo._base.lang in this case) and type the following into your FireBug console:

>>> dojo.require("");
>>> var doctest = new dojox.testing.DocTest();
>>> doctest.run("dojo._base.lang");

It will then print out a line for each test in the FireBug console with the result and some info. In the before case it looks like this:

...
Test 44: OK: dojo.trim(" trim me ")
Test 45: OK: dojo.trim("\t\ttrim me ")
Test 46: OK: dojo.trim(" \t\t trim me \t ")
Test 47: OK: dojo.trim("trim me")
Test 48: OK: dojo.trim(" trim me")
Test 49: OK: dojo.trim("trim me ")
Test 50: OK: dojo.trim("")


Embed doctest
You can also embed the doctests into other test modules, i.e. if you are just want to add the doctests to your unittests and get them all as one result. You can simply check the property errors of your doctest instance:

>>> doctest.errors
[]

As you can see if there are no errors it’s length is 0.
You can see an example of how to do that here in the file data/tests/stores/QueryReadStore.js the first function testDocTests() runs the doctests of the module “dojox.data.QueryReadStore” and verifies that there are no errors.

I hope this will be useful to some people. Let me know if you have any kind of feedback, bugs or whatever. In the class itself you can see that I have also already thought of some things that need to be done/added to this class. Have fun doctesting …