<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; "><BR><DIV><DIV>On Jun 5, 2005, at 10:22 PM, Ryan C. Gordon wrote:</DIV><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I suspect Stephane is going to rip the UI a new asshole for 2.0, so I spent some time thinking about the plugins instead.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Overall, I'm pleased with how the 1.0 archive system worked out in principle, but in implementation it has a lot of the same problems as the UI layer: code duplication, assumptions, and generic higher-level burdens. There are various generic tags that each plugin is required to cut and paste from existing plugins, and some do and some don't...plus there are UI callbacks passed around, etc.</DIV></BLOCKQUOTE><DIV><BR class="khtml-block-placeholder"></DIV><DIV>I agree. While the plugin system we have now mostly works, its API has become way too encumbered because of repeated changes to the file copying code, to which it is currently tied.</DIV><BR><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">It is exceedingly difficult to implement a new plugin, even for people that have done it before. When a new XML attribute is added to the codebase, it requires editing each plugin to support generic functionality. Working with the plugin code tends to be error-prone in this respect.</DIV></BLOCKQUOTE><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Agreed.</DIV><BR><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">So what we need, I think, is basically this:</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">1) Let's try to keep what's there intact in terms of functionality...we have perfectly good zip, tar, rar, etc handlers, it doesn't make sense to throw that existing code away entirely.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">2) Let's try to have all the high-level code exist in the base loki_setup, and make sure the plugins only deal with their archive. They should do one thing well.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">3) Let's have copying files and directory trees be a plugin so it's not a special case anymore.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">4) Let's get rid of the callbacks.</DIV></BLOCKQUOTE><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Sounds good. The callbacks in the plugin API, as I said, are a vestige of the old file-copying code, and used solely to provide a way for the UI to be updated while the files are being extracted. If we're going to rewrite the UI code, it makes sense to completely get rid of those anyway.</DIV><BR><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Basically, the plugins shouldn't be a one entry point "unpack this file and get back to me" thing. Ideally, they should just look more or less like the regular file code...the high level sees what files are there, and then copies them, block by block.</DIV></BLOCKQUOTE><DIV><BR class="khtml-block-placeholder"></DIV><DIV>This could be a valid approach. Of course, the rewrite of the plugin system will have to be done at the same time as a the rewrite of the whole I/O system, so we'll see exactly how this works out, but i like the idea of making regular files being handled as just another plugin. It will promote healthier code.</DIV><BR><BLOCKQUOTE type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">The plugin interface then looks something like this (naming convention is, of course, just an example here):</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">(these would be function pointers in a struct...)</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">First, find the plugin to handle a file by iterating over all plugins with this method...</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// Return non-zero if this is the .XXX plugin and this is really an .XXX</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// file. Where possible, plugin should determine this by file signature // (.zip files have end-of-central-directory, etc). Failing that, the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// plugin can report true or false based on the filename extension.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// In most cases, this would obsolete the "file" tag's "suffix"</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// attribute.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">int canHandleThisFile(const char *fname);</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Once you find the right plugin, open the archive...</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// prepare plugin to unpack this archive. Returns NULL on failure, an</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// opaque data type on success.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">void *openArchive(const char *fname);</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Once the archive is opened, you iterate through its contents with getNextItem() and readMoreData() until you are done...</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// Get the next item from the archive. It may be a file, symlink, dir,</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>or some other thing.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// PluginNextItemInfo is a struct like this:</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>struct {</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>const char *itemName;<SPAN class="Apple-converted-space"> </SPAN>// name of item ("path/myfile.txt", etc)</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>NextItemEnum itemType;<SPAN class="Apple-converted-space"> </SPAN>// file, dir, symlink, etc.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>size_t itemSize;<SPAN class="Apple-converted-space"> </SPAN>// total size of item (filesize, strlen of symlink)</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>};</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// Some types need no further processing, such as directories; the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>higher level will know what to do with that immediately.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// Other things (symlinks, files) will need more reading with</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//<SPAN class="Apple-converted-space"> </SPAN>readMoreData().</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// The plugin is expected to skip to the next valid item whether the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// current one was read or not, since the higher-level may choose to</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// filter a given item out of the installation for various reasons (a</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// symlink on windows? XML tags introduce filtering capabilities?)</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// Returns 1 if there was another item, zero if EOF, -1 if error. Errors</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// should be considered fatal to the installation, EOF should be the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// normal loop termination condition.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">int getNextItem(void *handle, PluginNextItemInfo *nextItem);</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// Read more data from the current item...for symlinks, this will be</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// the name of the thing they point to, for files, this will be the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// uncompressed contents of the item. Data is stored at buf, a buffer</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// of (*bufsize) bytes. On return, the number of bytes retrieved will</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// be stored to (*bufsize), whether there was a problem or not.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">//</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// Returns 1 if there was data to read, zero if EOF, -1 if error. Errors</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// should be considered fatal to the installation, EOF should be the</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// normal loop termination condition. EOF only triggers when you try to</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// read and there are no more bytes. A read that only fills one byte</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// before hitting the EOF will still return 1, and the next call will</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// return zero. This allows for archivers to return less than a full</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// buffer each call if it's more convenient to their decompression</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">// algorithms.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">int readMoreData(void *handle, void *buf, size_t *bufsize);</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">...finally, clean up...</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">void closeArchive(void *handle);</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">...you'll notice that the 1.0 plugin API has two methods: Size and Copy.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Size is intentionally removed...this was always less than useful (as anyone that used it with a .tar.bz2 can tell you), and assumes you want to install the whole archive, which may not be true in the future if new XML tags show up to filter out items. Perhaps a better solution to this is to assume the total size of an install is 0 bytes unless explicitly stated using the existing XML attributes, and if the install doesn't turn out to match the expected size (0 or otherwise), pop up a message box saying so, and reporting the actual result, so that the developer always fills this in correctly during final testing before shipping the installer.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">The rest of the new API is, more or less, a broken-out version of the Copy() method, but with the control put into the caller's hands and expecting the plugin to do its work in smaller chunks, so the caller can handle the general tasks of updating the UI and doing the actual installing of files from one unified place.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Comments?</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">--ryan.</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV> <BR class="Apple-interchange-newline"></BLOCKQUOTE></DIV><BR><DIV> <P style="margin: 0.0px 0.0px 0.0px 0.0px"><FONT face="Helvetica" size="3" style="font: 12.0px Helvetica">--</FONT></P> <P style="margin: 0.0px 0.0px 0.0px 0.0px"><FONT face="Helvetica" size="3" style="font: 12.0px Helvetica">Stéphane Peter</FONT></P> <P style="margin: 0.0px 0.0px 0.0px 0.0px"><FONT face="Helvetica" size="3" style="font: 12.0px Helvetica"><A href="mailto:megastep@megastep.org">megastep@megastep.org</A></FONT></P> </DIV><BR></BODY></HTML>