<?xml version="1.0" standalone="no"?>
-<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.1//EN"
- "http://www.oasis-open.org/docbook/xml/4.1/docbook.dtd">
-
-<refentry id="mimedecode.py">
+<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+ "file:///usr/share/xml/docbook/schema/dtd/4.5/docbookx.dtd">
+
+<refentry id="mimedecode">
+
+<refentryinfo>
+ <title>mimedecode</title>
+ <productname>mimedecode.docbook</productname>
+ <author>
+ <firstname>Oleg</firstname>
+ <surname>Broytman</surname>
+ <email>phd@phdru.name</email>
+ <personblurb/>
+ </author>
+ <copyright>
+ <year>2001-2018</year>
+ <holder>PhiloSoft Design.</holder>
+ </copyright>
+</refentryinfo>
<refmeta>
- <refentrytitle>mimedecode.py</refentrytitle>
+ <refentrytitle>mimedecode</refentrytitle>
<manvolnum>1</manvolnum>
</refmeta>
<refnamediv>
- <refname>mimedecode.py</refname>
+ <refname>mimedecode</refname>
<refpurpose>decode MIME message</refpurpose>
</refnamediv>
<refsynopsisdiv>
<cmdsynopsis>
- <command>mimedecode.py</command>
+ <command>mimedecode</command>
<arg choice="opt">
<option>-h|--help</option>
</arg>
<option>-f charset</option>
</arg>
<arg choice="opt">
- <option>-d header</option>
+ <option>-H|--host=hostname</option>
+ </arg>
+ <arg choice="opt">
+ <option>-d header1[,header2,header3...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-d *[,-header1,-header2,-header3...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-p header1[,header2,header3,...]:param1[,param2,param3,...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-p *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-p header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-p *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-r header1[,header2,header3...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-r *[,-header1,-header2,-header3...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-R header1[,header2,header3,...]:param1[,param2,param3,...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>-R *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</option>
</arg>
<arg choice="opt">
- <option>-p header:param</option>
+ <option>-R header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</option>
</arg>
<arg choice="opt">
- <option>-beit mask</option>
+ <option>-R *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</option>
+ </arg>
+ <arg choice="opt">
+ <option>--set-header header:value</option>
+ </arg>
+ <arg choice="opt">
+ <option>--set-param header:param=value</option>
+ </arg>
+ <arg choice="opt">
+ <option>-BbeIit mask</option>
+ </arg>
+ <arg choice="opt">
+ <option>--save-headers|body|message mask</option>
+ </arg>
+ <arg choice="opt">
+ <option>-O dest_dir</option>
</arg>
<arg choice="opt">
<option>-o output_file</option>
<title>DESCRIPTION</title>
<para>
Mail users, especially in non-English countries, often find that mail
-messages arrived in different formats, with different content types, in
-different encodings and charsets. Usually it is good because it allows to use
-an appropriate format/encoding/whatever. Sometimes, though, some unification is
-desirable. For example, one may want to put mail messages into an archive,
-make HTML indices, run search indexer, etc. In such situations converting
-messages to text in one character set and skipping some binary attachments is
-much desirable.
+ messages arrived in different formats, with different content types, in
+ different encodings and charsets. Usually it is good because it allows to
+ use an appropriate format/encoding/whatever. Sometimes, though, some
+ unification is desirable. For example, one may want to put mail messages
+ into an archive, make HTML indices, run search indexer, etc. In such
+ situations converting messages to text in one character set and skipping
+ some binary attachments is much desirable.
</para>
<para>
- Here is the solution - mimedecode.py!
+ Here is a solution - mimedecode!
</para>
<para>
- It is a program to decode MIME messages. The program expects one input file
-(either on the command line or on stdin) which is treated as an RFC822 message,
-and decoded to stdout. If the file is not an RFC822 message it is just piped to
-stdout as is. If the file is a simple RFC822 message it is just decoded as one
-part. If it is a MIME message with multiple parts ("attachments") all parts are
-decoded recursively. Decoding can be controlled by the command-line options.
+ This is a program to decode MIME messages. The program expects one input
+ file (either on command line or on stdin) which is treated as an RFC822
+ message, and decodes to stdout or an output file. If the file is not an
+ RFC822 message it is just copied to the output one-to-one. If the file is a
+ simple RFC822 message it is decoded as one part. If it is a MIME message
+ with multiple parts ("attachments") all non-multipart subparts are decoded.
+ Decoding can be controlled by the command-line options.
</para>
<para>
- First, Subject and Content-Disposition headers are examined. If any of those
-exists, it is decoded according to RFC2047. Content-Disposition header is not
-decoded - only its "filename" parameter. Encoded header parameters violate
-the RFC, but widely deployed anyway, especially in the M$ Ophice GUI (often
-referred as "Windoze") world, where programmers are often ignorant lamers who
-never even heard about RFCs. Correct parameter encoding specified by RFC2231.
-This program decodes RFC2231-encoded parameters, too.
+ First, for every part the program removes headers and parameters listed with
+ -r and -R options. Then, Subject and Content-Disposition headers (and all
+ headers listed with -d and -p options) are examined. If any of those exists,
+ they are decoded according to RFC2047. Content-Disposition header is not
+ decoded (if it was not listed in option -d) - only its "filename" parameter.
+ Encoded header parameters violate the RFC, but widely deployed anyway by
+ ignorant coders who never even heard about RFCs. Correct parameter encoding
+ specified by RFC2231. This program decodes RFC2231-encoded parameters, too.
</para>
<para>
Then the body of the message (or the current part) is decoded. Decoding
-starts with looking at header Content-Transfer-Encoding. If the header
-specifies non-8bit encoding (usually base64 or quoted-printable), the body
-converted to 8bit. Then, if its content type is multipart (multipart/related or
-multipart/mixed, e.g) every part is recursively decoded. If it is not
-multipart, mailcap database is consulted to find a way to convert the body to
-plain text. (I have no idea how mailcap could be configured on said M$ Ophice
-GUI, please don't ask me; real OS users can consult my example at
-http://phdru.name/Software/dotfiles/mailcap.html). The decoding process uses
-first copiousoutput filter it can find. If there is no any filter the body just
-passed unconverted.
+ starts with looking at header Content-Transfer-Encoding. If the header
+ specifies non-8bit encoding (usually base64 or quoted-printable), the body
+ is converted to 8bit (can be prevented with -B). Then if its content type
+ is multipart (multipart/related or multipart/mixed, e.g) every part is
+ recursively decoded. If it is not multipart, mailcap database is consulted
+ to find a way to convert the body to plain text (can be prevented with
+ options -Bbei). (The author has no idea how mailcap can be configured on
+ OSes other than POSIX, please don't ask; users can consult an example at
+ <ulink url="https://phdru.name/Software/dotfiles/mailcap.html">https://phdru.name/Software/dotfiles/mailcap.html</ulink>).
+ The decoding process uses the first copiousoutput filter it can find. If
+ there are no filters the body just passed as is.
</para>
<para>
- Then Content-Type header consulted for charset. If it is not equal to
-current default charset the body text recoded. Finally message headers and body
-flushed to stdout.
+ Then Content-Type header is consulted for charset. If it is not equal to the
+ current locale charset and recoding is allowed (see options -Cc) the body
+ text is recoded. Finally message headers and the body are flushed to stdout.
</para>
</refsect1>
+<refsect1>
+ <para>
+ Please be reminded that in the following options asterisk is a shell
+ metacharacter and should be escaped or quoted. Either write -d \*,-h1,-h2
+ or -d '*,-h1,-h2' or such.
+ </para>
+</refsect1>
<refsect1>
<title>OPTIONS</title>
<term>-c</term>
<listitem>
<para>
- Recode different character sets in message body to current default
- charset; this is the default.
+ Recode different character sets in message bodies to the current
+ default charset; this is the default.
</para>
</listitem>
</varlistentry>
<term>-C</term>
<listitem>
<para>
- Do not recode character sets in message body.
+ Do not recode character sets in message bodies.
</para>
</listitem>
</varlistentry>
<term>-f charset</term>
<listitem>
<para>
- Force this charset to be the current default charset instead of
- sys.getdefaultencoding().
+ Force this charset to be used for recoding instead of charset from
+ the current locale.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-H hostname</term>
+ <term>--host=hostname</term>
+ <listitem>
+ <para>
+ Use this hostname in X-MIME-Autoconverted headers instead of the
+ current hostname.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-d header1[,header2,header3...]</term>
+ <listitem>
+ <para>
+ Add the header(s) to a list of headers to decode; initially the
+ list contains headers "From", "To", "Cc", "Reply-To",
+ "Mail-Followup-To" and "Subject".
</para>
</listitem>
</varlistentry>
<varlistentry>
- <term>-d header</term>
+ <term>-d *[,-header1,-header2,-header3...]</term>
<listitem>
<para>
- Add the header to a list of headers to decode; initially the list
- contains headers "From" and "Subject".
+ This variant completely changes headers decoding. First, the list of
+ headers to decode is cleared (as with -D). Then all the headers are
+ decoded except the given list of exceptions (headers listed with
+ '-'). In this mode it would be meaningless to give more than one -d
+ options but the program doesn't enforce the limitation.
</para>
</listitem>
</varlistentry>
</varlistentry>
<varlistentry>
- <term>-p header:param</term>
+ <term>-p header1[,header2,header3,...]:param1[,param2,param3,...]</term>
<listitem>
<para>
- Add the (header, param) pair to a list of headers' parameters to
- decode; initially the list contains header "Content-Disposition",
- parameter "filename".
+ Add the parameter(s) to a list of headers parameters to decode;
+ the parameter(s) will be decoded only for the given header(s).
+ Initially the list contains header "Content-Type", parameter "name";
+ and header "Content-Disposition", parameter "filename".
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-p *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</term>
+ <listitem>
+ <para>
+ Add the parameter(s) to a list of headers parameters to decode;
+ the parameter(s) will be decoded for all headers except the given
+ ones.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-p header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</term>
+ <listitem>
+ <para>
+ Decode all parameters except listed for the given list of headers.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-p *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</term>
+ <listitem>
+ <para>
+ Decode all parameters except listed for all headers (except listed).
</para>
</listitem>
</varlistentry>
<term>-P</term>
<listitem>
<para>
- Clear the list of headers' parameters to decode (make it empty).
+ Clear the list of headers parameters to decode (make it empty).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-r header1[,header2,header3...]</term>
+ <listitem>
+ <para>
+ Add the header(s) to a list of headers to remove completely;
+ initially the list is empty.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-r *[,-header1,-header2,-header3...]</term>
+ <listitem>
+ <para>
+ Remove all headers except listed.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-R header1[,header2,header3,...]:param1[,param2,param3,...]</term>
+ <listitem>
+ <para>
+ Add the parameter(s) to a list of headers parameters to remove;
+ the parameter(s) will be removed only for the given header(s).
+ Initially the list is empty.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-R *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</term>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-R header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</term>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-R *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</term>
+ <listitem>
+ <para>
+ Remove listed parameters (or all parameters except listed) from
+ these headers (or from all headers except listed).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>--set-header header:value</term>
+ <listitem>
+ <para>
+ The program sets or changes value for the header to the given value
+ (only at the top-level message).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>--set-param header:param=value</term>
+ <listitem>
+ <para>
+ The program sets or changes value for the header's parameter to the
+ given value (only at the top-level message). The header must exist.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-B mask</term>
+ <listitem>
+ <para>
+ Append mask to the list of binary content types that will be not
+ content-transfer-decoded (will be left as base64 or such).
</para>
</listitem>
</varlistentry>
<listitem>
<para>
Append mask to the list of binary content types; if the message to
- decode has a part of this type the program will pass the part as is,
- without any additional processing.
+ decode has a part of this type the program content-transfer-decodes
+ (base64 or whatever to 8bit binary) it and outputs the decoded part
+ as is, without any further processing.
</para>
</listitem>
</varlistentry>
<listitem>
<para>
Append mask to the list of error content types; if the message to
- decode has a part of this type the program will raise ValueError.
+ decode has a part of this type the program fails with ValueError.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>-I mask</term>
+ <listitem>
+ <para>
+ Append mask to the list of content types to completely ignore.
+ There will be no output - no headers, no body, no warning. For a
+ multipart part the entire subtree is removed.
</para>
</listitem>
</varlistentry>
<term>-i mask</term>
<listitem>
<para>
- Append mask to the list of content types to ignore; if the message to
- decode has a part of this type the program will not pass it, instead
- a line "Message body of type `%s' skipped." will be issued.
+ Append mask to the list of content types to ignore; if the message
+ to decode has a part of this type the program outputs headers but
+ skips the body. Instead a line "Message body of type %s skipped."
+ will be issued.
</para>
</listitem>
</varlistentry>
<listitem>
<para>
Append mask to the list of content types to convert to text; if the
- message to decode has a part of this type the program will consult
- mailcap database, find first copiousoutput filter and convert the
- part.
+ message to decode has a part of this type the program consults
+ mailcap database, find the first copiousoutput filter and, if any
+ filter is found, converts the part.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>--save-headers mask</term>
+ </varlistentry>
+
+ <varlistentry>
+ <term>--save-body mask</term>
+ </varlistentry>
+
+ <varlistentry>
+ <term>--save-message mask</term>
+ <listitem>
+ <para>
+ Append mask to lists of content types to save to files;
+ --save-headers saves only decoded headers of the message (or
+ the current subpart); --save-body saves only decoded body;
+ --save-message saves the entire message or subpart (headers +
+ body).
</para>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>-O dest_dir</term>
+ <listitem>
+ <para>
+ Set destination directory for the output files; if the directory
+ doesn't exist it will be created. Default is the current directory.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term>-o output_file</term>
<listitem>
<para>
- Useful to set the output file in case of redirected stdin:
- <programlisting language="sh">mimedecode.py -o output_file < input_file
-cat input_file | mimedecode.py -o output_file</programlisting>
+ Save output to the file related to the destination directory from
+ option -O. Also useful in case of redirected stdin:
+ <programlisting language="sh">mimedecode -o output_file < input_file
+cat input_file | mimedecode -o output_file</programlisting>
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
- The last 4 options (-beit) require more explanation. They allow a user
-to control body decoding with great flexibility. Think about said mail
-archive; for example, its maintainer wants to put there only texts, convert
-Postscript/PDF to text, pass HTML and images as is, and ignore everything
-else. Easy:
+ The 5 list options (-Bbeit) require more explanation. They allow a user to
+ control body decoding with great flexibility. Think about said mail archive;
+ for example, its maintainer wants to put there only texts, convert
+ PDF/Postscript to text, pass HTML and images decoding base64 to html but
+ leaving images encoded, and ignore everything else. This is how it could be
+ done:
</para>
<para>
<code language="sh">
- mimedecode.py -t application/postscript -t application/pdf -b text/html
- -b 'image/*' -i '*/*'
+ mimedecode -t application/pdf -t application/postscript -t text/plain
+ -b text/html -B 'image/*' -i '*/*'
</code>
</para>
<para>
- When the program decodes a message (or its part), it consults
-Content-Type header. The content type is searched in all 4 lists, in order
-"text-binary-ignore-error". If found, appropriate action performed. If not
-found, the program search the same lists for "type/*" mask (the type of
-"text/html" is just "text"). If found, appropriate action performed. If not
-found, the program search the same lists for "*/*" mask. If found,
-appropriate action performed. If not found, the program uses default
-action, which is to decode everything to text (if mailcap specifies
-a filter).
+ When the program decodes a message (non-MIME or a non-multipart subpart of a
+ MIME message), it consults Content-Type header. The content type is searched
+ in all 5 lists, in order "text-binary-ignore-error". If found, appropriate
+ action is performed. If not found, the program searches the same lists for
+ "type/*" mask (the type of "text/html" is just "text"). If found,
+ appropriate action is performed. If not found, the program searches the same
+ lists for "*/*" mask. If found, appropriate action is performed. If not
+ found, the program uses the default action, which is to decode everything to
+ text (if mailcap specifies a filter). This algorithm allows more specific
+ content types to override less specific: -b image/* will be processed
+ earlier than -B */*.
+</para>
+
+<para>
+ Options -e/-I/-i can also work with multipart subparts of a MIME message. In
+ case of -I/-i the entire subtree of that multipart is removed; with -i it's
+ replaced with ignore warning.
+</para>
+
+<para>
+ Initially all 5 lists are empty, so without any additional parameters
+ the program always uses the default decoding (as -t */*).
+</para>
+
+<para>
+ The 3 save options (--save-headers/body/message) are similar. They make the
+ program to save every non-multipart subpart (only headers, or body, or the
+ entire subpart: headers + body) that corresponds to the given mask to a file.
+ Before saving the message (or the subpart) is decoded according to all other
+ options and is placed to the output stream as usual. Filename for the file is
+ created using "filename" parameter from the Content-Disposition header, or
+ "name" parameter from the Content-Type header if one of those exist; a serial
+ counter is prepended to the filename to avoid collisions; if there are no
+ name/filename parameters, or the name/filename parameters contain forbidden
+ characters (null, slash, backslash) the filename is just the serial counter.
</para>
<para>
- Initially all 4 lists are empty, so without any additional parameters
-the program always uses the default decoding.
+ If the file doesn't have any extensions (no dots in the value of the
+ name/filename parameters, or the name is just the counter) the program tries
+ to guess an extension by looking up the content type in mime.types files
+ including .mime.types file in the user's home directory (if it exists). If
+ the file has an extension the program doesn't try to verify that it
+ corresponds to the content type.
+</para>
+
+<para>
+ The file is saved in the directory set with -O (default is the current
+ directory). The save options are proceeded before -e options so the user can
+ save the message that causes an error.
</para>
</refsect1>
<refsect1>
<title>ENVIRONMENT</title>
+<variablelist>
+ <varlistentry><term>LANG</term></varlistentry>
+ <varlistentry><term>LC_ALL</term></varlistentry>
+ <varlistentry><term>LC_CTYPE</term></varlistentry>
+</variablelist>
<para>
- LANG
- LC_ALL
- LC_CTYPE
- Define current locale settings. Used to determine current default
- charset (if your Python is properly installed and configured).
+ Define current locale settings. Used to determine current default charset (if
+ your Python is properly installed and configured).
</para>
</refsect1>
<title>BUGS</title>
<para>
The program may produce incorrect MIME message. The purpose of the program
-is to decode whatever it is possible to decode, not to produce absolutely
-correct MIME output. The incorrect parts are obvious - decoded Subject headers
-and filenames. Other than that output is correct MIME message. The program does
-not try to guess whether the headers are correct. For example, if a message
-header states that charset is iso8859-5, but the body is actually in koi8-r -
-the program will recode the message to the wrong charset.
+ is to decode whatever it is possible to decode, not to produce absolutely
+ correct MIME output. The incorrect parts are obvious - decoded
+ From/To/Cc/Reply-To/Mail-Followup-To/Subject headers and filenames. Other
+ than that output is correct MIME message. The program does not try to guess
+ whether the headers are correct. For example, if a message header states
+ that charset is iso8859-1, but the body (HTML, for example) is actually in
+ utf-8 the program will recode the message with the wrong charset.
</para>
</refsect1>
<refsect1>
<title>AUTHOR</title>
<para>
- Oleg Broytman <phd@phdru.name>
+ <firstname>Oleg</firstname>
+ <surname>Broytman</surname>
+ <email>phd@phdru.name</email>
</para>
</refsect1>
<refsect1>
<title>COPYRIGHT</title>
<para>
- Copyright (C) 2001-2014 PhiloSoft Design
+ Copyright (C) 2001-2018 PhiloSoft Design.
</para>
</refsect1>
<refsect1>
<title>SEE ALSO</title>
<para>
- mimedecode.py home page: http://phdru.name/Software/Python/#mimedecode
+ mimedecode home page:
+ <ulink url="https://phdru.name/Software/Python/#mimedecode">https://phdru.name/Software/Python/#mimedecode</ulink>
</para>
</refsect1>
projects / mimedecode.git / blobdiff