[CLA-patch] Minimalist dojo build

[jbondc]

This patch is more of a proof of concept then something production ready.

Instead of including the whole *src* directory, a minimalist build will:

- compress dojo.js - scan the compressed dojo.js and look for 'ressources' or external dependencies (css, html, images, swf, etc...) and recurse over .css & .html to possibly find more ressources - intern strings

Basically only copy/release a 'minimum' set of files.

I noticed there's some new work being done on the build, hopefully this work can be looked at and officially integrated in some form.

[jburke]

Thank you for taking the time to do a patch. I've scheduled it for the 0.9 timeframe. I'm changing things quite a bit for the 0.9 build system, but I'll keep this approach in mind.

I am a bit concerned about catching all the files we should catch (for example .smd files for the json-rpc-derived modules). And, this patch is not quite production ready as you mentioned (for instance, templateDir is mentioned, but that does not show up anywhere in the dojo source, and the target/ method is called "copyRessources" instead of "copyResources"). I understand that you are trying to demonstrate the general approach.

[jbondc]

My mistake, I forgot to remove templateDir from the patch.

It's only there since I have some widgets with theme/styling support that use 'templateDir'

i.e. widget/foo.js templateDir: dojo.uri.dojoUri("src/widget/templates/ListItems"),

The build then catches this and copies all css under this directory: src/widget/templates/ListItems/blue.css src/widget/templates/ListItems/business.css

I haven't looked at dijit yet but I'm guessing something like this would be needed. There's definitely a lot of careful design work that needs to be done for the minimal approach to work. I'll be submitting other patches as time allows against more recent dojo revs

[alex]

James, given the recent work in this area, is this bug still apropos?

[jburke]

It still could be applicable, for someone that did not want to take all of the built dojo. However, I'll move it to 0.9, and perhaps have jbondc submit an updated patch after 0.9 beta to bring it inline with the 0.9 codebase. I'm hoping to have the build system fully fleshed out for 0.9beta, so it would be clearer how to make a patch.

[jburke]

jbondc: if you still think this might be useful for 0.9/1.0, feel free to provide a patch. I'm moving to 1.0, since I don't think this is a 0.9 blocker.

[guest]

Definetly, I'm slowly migrating to 0.9... I'll send a patch for 0.9 stable - 1.0

[guest]

Minimalist dojo build + WTI integration?

[guest]

This new patch allows dojo 0.9 to build a minimal release with only the required set of resources/files.

Patch implements "wti" which allows the dojo build to build javascript from other frameworks (such as Ext-Js, jquery, ...)

The idea is described here: http://jbondc.openmv.com/

An example profile: dependencies = {

layers: [

{

name: "../wti/integrated.js", dependencies: [

"dijit.layout.ContentPane?", "dijit.layout.TabContainer?", "jquery.interface.ifxscale", "jquery.interface.ifxshake", "Ext.ux.Accordion", "Ext.widgets.DatePicker?", "Ext.widgets.layout.ContentPanels?", "Ext.widgets.layout.BorderLayout?"

]

}

],

prefixes: [

[ "dijit", "../../dijit" ], [ "jquery", "../../jquery" ], [ "Ext", "../../Ext" ]

]

}

[guest]

OpenAjax? patch + fixes for YUI and add support for loading other resources like css?

[guest]

Hey James,

Noticed you checked in OpenAjax? so I'm keeping you on your toes :) First off this patch allow the dojo loader to work modified versions of jquery, Ext-js and YUI (openajax compliant?).

You can grab the modified YAHOO file(s) here: http://www.openmv.com/tests/YAHOO.zip

Some notes:

For some reason, YUI did not like: obj = (p in obj ? obj[p] : (create ? obj[p]={} : undefined));

i.e. "YAHOO.base" p in obj -- returned true while it should be false

I added an OpenAjax?.register method which essentially maps a resourceId to a URI. This in in conjunction with the 'registry' I think the OpenAjax? team is working on although it doesn't make any sense to me right now.

I have not modified the XD loader so maybe you can help fill in the blanks there.

I'll post more details about the work this weekend, enjoy

[jburke]

Jonathan: neat stuff! Some questions and comments:

1) I'm not sure we can add things to the OpenAjax? namespace, I'll have to check with Adam Peller (another contributor who is working on the OpenAjax? file that was recently committed to Dojo). He mentioned before that the group tried to work out a dependency mapping standard, but it got to be a bigger problem to sort out, so they started smaller with topics to get something out to start. It would be neat if we could piggyback on the OpenAjax? namespace, but if not, we might have to do some small renamings.

2) I notice you use the OpenAjax?.register to put in a mapping for the YUI module names to their resource names (the OpenAjax?.register call in YUI base.js). Did you prefer that over just using the file names as the dojo.provide()/require() strings? It is technically allowed that the require and provide strings used to load file resources are actually different than the modules they define.

If you used the file names for the resource strings, then you could avoid the base.js register call, as well as having to define OpenAjax?.register. I can appreciate though it is nice having the logical modules names to include in your code. Maybe instead, just rename the YUI files to match the modules they define, since with this system, the user should be using dojo.require to load them. I think that would make it easier all around. Your thoughts?

3) Extending require to handle resourceTypes: I can appreciate wanting a way to load CSS via a logical name (the dojo.require name) vs. having to put the link tag in yourself. But the behavior of link tags is a bit different than normal require modules (AFAIK, they can be asynchronously loaded), and there is not reliable way to know when they load. I've given some thought to it here: http://thread.gmane.org/gmane.comp.web.dojo.devel/7029

I think I was tending to a dojox module for doing the heavy "parse CSS and load via direct style tag" to get the sync loading and the ability to do i18n string replacements.

I'm also thinking it might be nice to reserve the extra args on dojo.require so that you can specify more than one JS dependency with one call: dojo.require("dojox.foo", "dojox.bar", "dojox.baz");

Maybe to support the different resource types, we have different methods. We could even look at doing a "light" dojo.requireCss that basically just does the module-to-path mapping and does a link tag, with no guarantees of loading. If we had other resource types we want to load, then we would have separate dojo.requireXXX methods for them. What do you think?

4) I'm probably missing it, but I don't see the code for doing the minimal work you were doing before (only pulling out resources that were used in the build profile). The minimal() function in the buildUtil.js patch section just seems to be a copy of the main build.js loop? Or did you intend the minimal stuff to be a separate patch (or did I just miss it?). I'm looking at build.3.patch.

5) I like the idea of using rhino to strip comments properly (part of the buildUtil.js, removeComments function patch). However, using Rhino for the web build system won't work. We could do an if/else thing for the web build, but since this comment removal is just part of the scanning for dojo.require/provide calls, it doesn't have to be perfect. Did you hit cases where the regexp caused a build problem (either not finding a dependency or finding one it should not have found)?

6) The YAHOO p in obj problem is an interesting find.

Some really neat stuff in here. Thanks for putting so much time into it.

[peller]

The OpenAjax? group has been talking about this type of feature, but it was too much for them to handle in their 1.0 hub. We should definitely float this by them. However, I think throwing something in their namespace would violate the namespace principles we share, just as we wouldn't want them to start adding their wish list to dojo.* :-)

If we want to propose something, would it make sense to use dojox as a temporary home? I know it doesn't have the generic ring to it that you're looking for.

Jonathan, do you want to approach them as an individual (not even sure if they figured out how that works in their org) or should we submit this request on behalf of Dojo, which is already a member?

[guest]

1) I agree that using OpenAjax? namespace for the require / provide mechanism doesn't seem right since its not official but then again putting the proof-of-concept out there could help things move along. What do you have in mind?

2) I was trying to finish writing up a detailed proposal yesterday but I ended up being short on time, I've put the work online here:

http://www.openmv.com/acme.pdf

3) Great stuff! It sounds like the problem is more with the ally code, I think any loadCss function should not be expected to know when the CSS will be loaded. It's only cosmetic styling that is expected to be applied by the browser.

I agree with using dojo.requireXXX methods, after some rethinking I got rid of the resourceType and moved that info into the registry. You can see how the concept is applied here: http://www.openmv.com/acme/base.js

What do you guys think of starting a new dojox.loader project? Basically create a loader that can load javascript and css based on information from a registry.

4) Right, I haven't moved over any of the copying the resources code since I haven't tackled the CSS loader yet. That's step 2 which I was planning to start end of next week. The minimal function right now only copies the compiled javascript which is useless just by itself.

5) Ya I ran into an issue with strings containing a comment: "// fdfdsfdsfd"; The other types of comments could also be problem var a = " hghfgfgh /* ";

.. somewhere else

var b = " */ ";

Even if the odds of this are ^$#!?

Let me know if starting a dojox.loader project sounds right, I'd be happy to work on it

As for OpenAjax?, I started working on a proposal -- i think i'll just send it their along with a member agreement

[jburke]

Jonathan: I think we're going to try to wrap up the 1.0 bugs soon. If you want to submit a current patch for the minimalist build, I'll give a review to see if we can include it for 1.0.

For the other loader things that you want to explore, a dojox.loader project might be a good idea, and open separate ticket(s) for it. I can see where the dojo core will not load things from a registry, and I am not sure a registry based loading system makes sense in general for dojo, given the reasons I outlined in here: http://dojotoolkit.org/2007/08/29/yojo-loading-yui-dojo-loader

Some things I think would be useful, and something I hope to consider at some point, but feel free to take these on if you like:

What I would like to see is a build step that takes the registry info in YUI's loader file and automatically populate the YUI files with dojo.require/dojo/provide calls, so that we could use the normal dojo loader with the YUI code.

Additionally, have a build step we could run over the dojo code that would build a set of calls that could be registered with the YUI loader, so they could use their registry-based system to load Dojo code.

If you are interested in those items, feel free to open a ticket on them and say you are working on them, so that I do not also try a parallel effort.

[jburke]

Need a current patch for this. Closing it for now, but feel free to reopen with a current patch for the minimalist build. May get pushed to 1.1 or 2.0 though.

[guest]

Hey James,

I'm still committed to this just have been quite busy, will resubmit a new ticket. Btw great work with YUI loading.