[mojosetup] Re: [lokisetup] Announcing MojoSetup.

[Ryan C. Gordon]

Sorry, this email got lost in my inbox, sorry to reply so late...

(Also, we should consider taking this conversation to the MojoSetup 
mailing list soon, but at this time, I'd like more feedback about these 
points from loki_setup users.)

> Well, there was a dtd file for setup.xml you could enforce pretty
> much everything too. That's one of the main features of xml.

Yes, but there are places in the loki_setup source that can't be 
validated: places that check for "yes", "true", "on", or just an 
attribute's existence at all. In these cases, a DTD is not really 
helpful (although to be fair, these are actually misuses of libxml in 
loki_setup, which provides an API for checking boolean attributes and 
covers all those different values).

My thought is that a DTD adds more complexity, which Lua, or any other 
scripting language, gives you intrinsically.

> Not that I am a big fan of xml but I bet you could write a
> not-so-generic, non-validating xml parser with less code than
> libxml2.

There were definitely other solutions, to be sure, such as tinyxml, but 
I liked the extra functionality of Lua...it gets you scripting and a 
data definition language in one.

Someone else pointed out "json" to me, and I think that could have been 
interesting for the same reasons...plus Javascript is more widespread 
(and honestly not bad in scenarios like this where you don't have to 
fight with incompatible web browsers)...but I happened to like Lua and 
it was extremely nice for embedding in ways that maybe Seamonkey 
wouldn't have been if I had researched it more.

> ... which means it won't be used. You'd really need to enforce
> vendors to use such a package format. However, as long as you allow
> vendors to 'customize' the installer by hacking the 'engine' and
> ship such modified binaries any attempt to have a package format
> that can be installed with an external application will fail.

I was mostly just hypothesizing...I don't expect distros to pick up 
support for MojoSetup as a package format, and it probably works better 
if they don't, since it gives ISVs more freedom to do as they like 
within their own download.

> Well, if you decide to add a statically linked mojosetup gtk1 GUI
> you'd have the same problem as with loki_setup :-)

True, but there's still a win here in that the GUI interface is small 
and self-contained. Even if you split all the installer logic out of the 
various loki_setup GUI sources, you still end up with a full copy of the 
installer for each binary. Pushing these into shared libraries was meant 
to minimize the bulk, even if GTK1 gets statically linked to one of 
them...but I'm also hoping to find that having each library be so small 
can make even the worst-case scenario (multiple copies of the 
otherwise-same GUI plugin for various binary incompatible distros) not 
so bad that we have to resort to static linking. We'll see if that works 
out in practice, though.

> Please, please don't. Just resist the wxWidgets trolls. KDE also
> doesn't need a separate gui. GTK2 will just work fine. 99.99% have

Sure, it'll work fine, but the hope is to make the UI consistent with 
the rest of their system. If adding a Qt UI makes the download 10 or 20k 
larger, I think that's an acceptable tradeoff, even if they would have 
just fallen back to the GTK+ UI anyhow.

(I also happen to like wxWidgets, but that's a discussion for another 
day. Generally in a project like this that can target multiple UIs 
easily enough, there isn't a serious value to it.)

> that installed anyways due to Firefox etc. A C++ GUI would only work
> when provided by the system rather than the downloaded installer due
> to binary compatability issues. Unless you ship the C++ libs as well
> which kind of defeates the purpose of your plugin system.

This was part of the reason for the plugin system: if the C++-using 
plugin isn't binary compatible, the installer will just go on to another 
one when the dlopen() fails. The base installer itself is strictly C, 
though, to avoid such issues.

> I don't think it's a good idea to give lua scripts to translators.
> There are very good tools to aid translating po files, like kbabel
> or emacs. You could probably just compile and merge the .po files
> you get from translators into a lua script for the actual installer
> though. SUSE uses a mechanism like that to have .desktop files for
> the distribution translated for example.

Someone is welcome to write such a translation tool, but my experience 
is that being able to hand out a single text file to the translators is 
extremely convenient...but I recognize there are different requirements 
between, say, what a Linux distro wants and what Google Earth's 
translation team wants, though. MojoSetup is targeted at ISVs, many of 
which have no idea what a .po file is or have convenient access to tools 
for them, but every Windows box ships with Notepad.exe, so passing one 
text file around by email seems like the least threatening approach to 
translators who aren't programmers and might be non-technical people in 
general.

> I guess vendors find that totally sexy. IMO installers that
> automatically download crap from the internet themselves are
> horrible. I want to be able to install that thing I downloaded
> yesterday to also work tomorrow when the vendor's web server went
> out of service or I don't have an internet connection (LAN party!).
> Also if you want to install the application on more than one system
> it sucks to have the installer download it multiple times.

Generally I agree with you, but I could see scenarios where it can be 
VERY useful. Casual titles still take a beating on their attach rate if 
the initial download takes too long, as they still have a lot of 
non-hardcore users on dial-up...for them, it could be very useful to get 
the user into a GUI as quickly as possible so they don't hit "cancel" in 
the browser download window, and do the rest of the download once they 
feel invested in the process.

Also, it could be useful for things that have large packages that a user 
never installs...Microsoft's Platform SDK does this, for example; why 
download 300 megabytes of Itanium tools if you only want the i386 
compiler? In these cases, it makes the total bandwidth used 
significantly less.

I don't think you're wrong overall, though: I don't intend to use the 
feature for any immediate projects, but felt that designing it in made a 
lot of sense for some not-so-uncommon conditions.

> I'd also make some sort of authenticity check mandatory in order to
> prevent man in the middle attacks. Ie enforce specification of md5
> or sha1 sums for files in the config file.

The installer doesn't do this now, but it's on the TODO list, both for 
complete archives (download or otherwise), and stuff coming from 
physical media (catch that reissued disc where the publisher made 
unexpected changes, for example).

> Means you can't even compile mojosetup on the pretty recent openSUSE
> 10.2 due to some macro package that is not yet in cmake 2.4.3 but
> only in 2.4.6 ... o_O

That's a problem that solves itself as KDE development moves forward, 
since a lot of good improvements to CMake landed in 2.4.6 due to KDE 
switching to it. Distros will upgrade; I don't really care what OpenSUSE 
10.2 does, because sooner or later OpenSUSE 11 will ship.

CMake solves so many problems, it's worth the initial wait for distros 
to catch up (or the interim hassle of installing it from source). I am 
_never_ going back to autoconf/automake, and I'm switching several of my 
other projects to use CMake as well.

I evaluated several other build systems, such as scons, and CMake fit my 
needs the best.

> For example a database of installed applications and files and an
> update mechanism. At least the former is a shop stopper feature
> IMHO.

Yeah, that's definitely on the TODO list. It's not hard to do, it just 
hasn't been done yet.

A lot of the immediate improvements to MojoSetup are going to come from 
seeing what people that are building installers need. For example, TTimo 
already sent in a large pile of issues, and most are ironed out already.

There are still some glaring omissions (like the inability to launch a 
web browser or the newly-installed app), but improvements are 
prioritized by what people say they need most.

> After a glance at the source code it looks like you also allow shell
> script hooks. Wouldn't it be better to enfoce lua scripts instead?
> With lua I guess you could intercept file modifications so vendors
> can't do arbitrary stupid things behind the installer's back.

At the moment, there are hooks to call config-file provided Lua 
functions, not shell scripts. Some of the loki_setup docs are 
cut-and-pasted into a comment right now, though, so I could see what 
features I should be thinking about, and one might mention shell scripts.

I have a feeling you can't really ever entirely prevent arbitrary stupid 
things, but in the case of Lua, we could at least enforce sandboxing.

For ISVs shipping their own installers, it gives you the whole Lua 
runtime by default (including the ability to spawn processes and access 
the filesystem). If you're going to run an untrusted binary to start 
with, I don't think adding an artificial limitation is reasonable, but 
if this becomes something the distros control, sandboxing is easy enough 
to enable. This isn't nearly close to the point where we should be 
talking about formalizing the installation process in ways we would need 
to if distros provided their own MojoSetup binary for third-party 
packages, though.

> I noticed that you include zlib and bzip2 source code. Is that
> really necessary? Even if you want to link them statically you can
> just take the system provided libraries.

Precautions for systems that don't have them, and in the case of zlib, 
so we can rebuild them with -DZ_PREFIX=1 if statically linking.

That's just my preference for packaging the source...I like that the 
build system can recover, without bothering the developer, on systems 
without a bzlib2 dev package, even if Linux developers disapprove in 
principle.

--ryan.