Why sometimes the patch you downloaded for UT2k3 Breaks,
        and how to make it fixed.


Imagine you have a file.

The top part of the file is shell script.

The bottom part of the file is binary data.

It could be, for example, that you're using the rocking makeself
http://www.megastep.org/makeself/ , part of setup [which you want],
http://icculus.org/loki_setup/ . To use the file, "sh file.run".

Except that first, you have to get it onto the machine you wish to run
it on. This turns out to be a non-trivial task, and hard to trace when
stuff breaks, since the error messages you get are NOT helpful.

IE and Outlook are prime perpetrators of dangerous stupidity, but it
should be noted that Mozilla and friends aren't exempt. Do not trust
e-mail programs. Most bloody definitely do not trust web browsers.


Now. Imagine you give the file any of the following suffixes:
.run .clp
[just examples, there]

The three-letter extension on these files isn't recognised by most
webservers as they come in the box, so in order to determine mime-type
before sending the file to the client, the web servers run some magic
on the files to determine their type.

When running magic on these files, you get one of two types out:

Content-type: application/x-sh
If the magic detection spots the #!/bin/sh at the top of the file
[this would be what Apache tends to dig out], or:

Content-type: text/plain
If the magic detection merely notices that the top 3 or 4 k of the
file is raw ascii.
[all you poor people using IIS, this is the one]


If a file is downloaded as either of these types by most of the web
browsers out there [including, notably, all versions of IE, netscape 4,
and mozilla], then the browser actually "helpfully" changes the
newlines in the file for you.

This is because they're currently unix newlines which doesn't sit well
with most windows-based apps.

Two bad things then happen:
1) The shell script no longer functions as a shell script because every
line has a garbage character at the end
2) In the binary data of the tarball, things that look like newlines
are converted, which also causes damage to the file.

This is a non-recoverable problem, which requires a re-download.


The challenge for today is to serve them up correctly first time.

First, most simply, ensure than the Content-type is
"application/octet-stream". It's not really ideal in a lot of situations,
but it's the one mime type that all browsers understand as
"Do not f**king munge. Save to disk as-is".

But how?
If you're serving the file statically [ie, it's just a file on-disk],
then you simply need to explain to your web server how it works:

If you're using Apache:
Edit your httpd.conf file, and add this to the <IfModule mod_mime.c>
section
AddType application/octet-stream .run .clp
Then give the server an "apachectl graceful" and it'll pick that
change up.

If you're using IIS:
http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/windowsserver2003/proddocs/server/wsa_mimemapcfg.asp

No, I am not competent to answer questions on IIS.

All of which is wonderfully simple.


If you're dynamically generating these files [yours truly, here], then
you'll need to just add some headers to your HTTP response before sending
the file out. If you're using some random, flat, langauge [perl, bourne],
just print out "Content-type: application/octet-stream" and a couple
carraige returns before printing out the file.
If you're using php, then use the "header" command:
header("Content-type: application/octet-stream");
That must be before you print out anything else.


For more funky bonus points, if you want the browser to have a
working progress bar, you need to tell it the size of the file they're
downloading:
header("Content-length: " . strlen($content));
or
header("Content-length: " . (string)(filesize($filename)));


For reference at this point, if you need to tell the browser what filename
to save as, there are several things you can do:

1) Make sure the URL ends in /filename.run. This seems like the easiest
one to do, but it's just a bitch in practice. The web server understands
certain things, here:
http://foo.bar.org.uk/serve-file.php/filename.run
Will actually execute serve-file.php, and the browser will think it's
called "filename.run". [For bonus points, your php script can make use
of that extra metadata, too. W00t.]

It seems that some distributions don't have this working right. Fix
your server.

2) Serve the header
Content-disposition: attachment; filename=filename.run
and the browser should recognise "filename.run" as the filename.

Beware that some browsers [grrr] ignore the Content-disposition header
if they think they're cleverer than the server, and show you the file.

Beware, also, that some things don't support this. CUPS, you suck ;-P


W00t. You now know how to serve files up, webby-wise, so that browsers
will handle them correctly. But what if you want to send the file in
an e-mail? Here we move onto content encodings, too. Joy.

I have a feel this is one of those "some code is worth many-many words"
moments [php, I'm afraid]:


$to = "Chunky Kibbles <chunky@icculus.org>";
$from = "Chunky's only Friend <chunky@icculus.org>";
$subject = "Test e-mail";
$filename = "fooingbar.run";

$handle = fopen($filename,"rb"); // Fucking windows caring what filetype
                                 //   something is. This would be half the
                                 //   bloody problem.
$file_content = fread($handle,filesize($filename));
$base64_content = chunk_split(base64_encode($file_content));
fclose($handle);

$boundary = md5("Chunky Kibbles");

/* Headers here: */

$email_headers = <<<EOHEADER
From: $from
MIME-Version: 1.0
Content-Type: multipart/mixed;
        boundary=$boundary
EOHEADER;


/* Content here: */

$email_content = <<<EOCONTENT
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

Put some textual helpful content here


--$boundary
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=$filename
Content-Transfer-Encoding: base64

$base64_content

--$boundary--
EOCONTENT;

/* Note the terminating --$boundary-- */


mail($to,$subject,$email_content,$email_headers,"-f $from");

/* You know your language is "too high-level" when it has a builtin
"mail" command */


Anyways. That will send a file, simply, in a manner understood by all
mail clients, and that outlook won't feel the urge to decimate.

That's a trivial example. If you're sending a big file, can I recommend
that you do the base64 encoding on-disk, and in chunks.


That's cool. Useful, even.
Now say that you want to send someone the content as plain text, but
if they have an HTML-capable MUA, then you can in some way add value to
your content:

Between any two $boundaries, you can put in some more
multipart/alternative bits:



--$boundary
Content-Type: multipart/alternative; boundary=$boundary_alt

--$boundary_alt
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit


Please pull the file foo.bar.run and anything else you need from
ftp://ftp.icculus.org/
using username & password:
username: foo
password: bar

Gary (-;
chunky@icculus.org

--$boundary_alt
Content-Type: text/html; charset="us-ascii"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

<P>Please pull the file foo.bar.run and anything else you need from
<A HREF="ftp://foo:bar@ftp.icculus.org/">ftp://ftp.icculus.org/</A>. If
you're asked for a password, please use:<BR>
username: foo<BR>
password: bar</P>

<P><A HREF="mailto:chunky@icculus.org"><ADDRESS>Gary (-;<BR>
&lt;chunky@icculus.org&gt;</ADDRESS></A></P>

--$boundary_alt--

/* Note the terminating --$boundary_alt-- */



Finally and since we're on the topic, I cannot make enough thank-you
noises to Primer, for all his help on stuff like this.

Gary (-;