mimedecode.docbook Oleg Broytman phd@phdru.name 2001-2014 PhiloSoft Design. mimedecode.py 1 mimedecode.py decode MIME message mimedecode.py -h|--help -V|--version -cCDP -f charset -H|--host=hostname -d header1[,header2,header3...] -d *[,-header1,-header2,-header3...] -p header1[,header2,header3,...]:param1[,param2,param3,...] -p *[,-header1,-header2,-header3,...]:param1[,param2,param3,...] -p header1[,header2,header3,...]:*[,-param1,-param2,-param3,...] -p *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...] -r header1[,header2,header3...] -r *[,-header1,-header2,-header3...] -R header1[,header2,header3,...]:param1[,param2,param3,...] -R *[,-header1,-header2,-header3,...]:param1[,param2,param3,...] -R header1[,header2,header3,...]:*[,-param1,-param2,-param3,...] -R *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...] --set-header header:value --set-param header:param=value -Bbeit mask -O dest_dir -o output_file input_file output_file Mail users, especially in non-English countries, often find that mail messages arrived in different formats, with different content types, in different encodings and charsets. Usually it is good because it allows to use an appropriate format/encoding/whatever. Sometimes, though, some unification is desirable. For example, one may want to put mail messages into an archive, make HTML indices, run search indexer, etc. In such situations converting messages to text in one character set and skipping some binary attachments is much desirable. Here is the solution - mimedecode.py! This is a program to decode MIME messages. The program expects one input file (either on command line or on stdin) which is treated as an RFC822 message, and decodes to stdout or an output file. If the file is not an RFC822 message it is just copied to the output one-to-one. If the file is a simple RFC822 message it is decoded as one part. If it is a MIME message with multiple parts ("attachments") all parts are decoded. Decoding can be controlled by command-line options. First, for every part the program removes headers and parameters listed with -r and -R options. Then, Subject and Content-Disposition headers (and all headers listed with -d and -p options) are examined. If any of those exists, they are decoded according to RFC2047. Content-Disposition header is not decoded - only its "filename" parameter. Encoded header parameters violate the RFC, but widely deployed anyway by ignorant coders who never even heard about RFCs. Correct parameter encoding specified by RFC2231. This program decodes RFC2231-encoded parameters, too. Then the body of the message (or the current part) is decoded. Decoding starts with looking at header Content-Transfer-Encoding. If the header specifies non-8bit encoding (usually base64 or quoted-printable), the body converted to 8bit. Then, if its content type is multipart (multipart/related or multipart/mixed, e.g) every part is recursively decoded. If it is not multipart, mailcap database is consulted to find a way to convert the body to plain text. (I have no idea how mailcap can be configured on OSes other than POSIX, please don't ask me; real OS users can consult my example at http://phdru.name/Software/dotfiles/mailcap.html). The decoding process uses the first copiousoutput filter it can find. If there are no filters the body just passed as is. Then Content-Type header is consulted for charset. If it is not equal to the current locale charset and recoding is allowed the body text is recoded. Finally message headers and the body are flushed to stdout. Please be warned that in the following options asterisk is a shell metacharacter and should be escaped or quoted. Either write -d \*,-h1,-h2 or -d '*,-h1,-h2' or such. -h -help Print brief usage help and exit. -V --version Print version and exit. -c Recode different character sets in message bodies to the current default charset; this is the default. -C Do not recode character sets in message bodies. -f charset Force this charset to be the current default charset instead of the current locale. -H hostname --host=hostname Use this hostname in X-MIME-Autoconverted headers instead of the current hostname. -d header1[,header2,header3...] Add the header(s) to a list of headers to decode; initially the list contains headers "From", "To", "Cc", "Reply-To", "Mail-Followup-To" and "Subject". -d *[,-header1,-header2,-header3...] This variant completely changes headers decoding. First, the list of headers to decode is cleared. Then all the headers are decoded except the given list of exceptions (headers listed with '-'). In this mode it would be meaningless to give more than one -d options but the program doesn't enforce it. -D Clear the list of headers to decode (make it empty). -p header1[,header2,header3,...]:param1[,param2,param3,...] Add the parameters(s) to a list of headers parameters to decode; the parameters will be decoded only for the given header(s). Initially the list contains header "Content-Type", parameter "name"; and header "Content-Disposition", parameter "filename". -p *[,-header1,-header2,-header3,...]:param1[,param2,param3,...] Add the parameters(s) to a list of headers parameters to decode; the parameters will be decoded for all headers except the given ones. -p header1[,header2,header3,...]:*[,-param1,-param2,-param3,...] Decode all parameters except listed for the given list of headers. -p *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...] Decode all parameters except listed for all headers (except listed). -P Clear the list of headers parameters to decode (make it empty). -r header1[,header2,header3...] Add the header(s) to a list of headers to remove completely; initially the list is empty. -r *[,-header1,-header2,-header3...] Remove all headers except listed. -R header1[,header2,header3,...]:param1[,param2,param3,...] Add the parameters(s) to a list of headers parameters to remove; the parameters will be decoded only for the given header(s). Initially the list is empty. -R *[,-header1,-header2,-header3,...]:param1[,param2,param3,...] -R header1[,header2,header3,...]:*[,-param1,-param2,-param3,...] -R *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...] Remove listed parameters (or all parameters except listed) frome these headers (or from all headers except listed). --set-header header:value The program sets or changes value for the header to the given value (only at the top-level message). --set-param header:param=value The program sets or changes value for the header's parameter to the given value (only at the top-level message). The header must exist. -b mask Append mask to the list of binary content types; if the message to decode has a part of this type the program will pass the part as is, without any additional processing. -B mask Append mask to the list of binary content types that will be not content-transfer-decoded (will be left as base64 or such). -e mask Append mask to the list of error content types; if the message to decode has a part of this type the program fails with ValueError. -i mask Append mask to the list of content types to ignore; if the message to decode has a part of this type the program will not pass it, instead a line "Message body of type `%s' skipped." will be issued. -t mask Append mask to the list of content types to convert to text; if the message to decode has a part of this type the program will consult mailcap database, find first copiousoutput filter and convert the part. -O dest_dir Set destination directory for the output files. Default is current directory. -o output_file Save output to the file related to the destination directory from option -O. Also useful in case of redirected stdin: mimedecode.py -o output_file < input_file cat input_file | mimedecode.py -o output_file The 5 list options (-Bbeit) require more explanation. They allow a user to control body decoding with great flexibility. Think about said mail archive; for example, its maintainer wants to put there only texts, convert Postscript/PDF to text, pass HTML and images as is, and ignore everything else. Easy: mimedecode.py -t application/postscript -t application/pdf -b text/html -b 'image/*' -i '*/*' When the program decodes a message (non-MIME or a non-multipart subpart of a MIME message), it consults Content-Type header. The content type is searched in all 4 lists, in order "text-binary-ignore-error". If found, appropriate action performed. If not found, the program search the same lists for "type/*" mask (the type of "text/html" is just "text"). If found, appropriate action performed. If not found, the program search the same lists for "*/*" mask. If found, appropriate action performed. If not found, the program uses default action, which is to decode everything to text (if mailcap specifies a filter). Initially all 4 lists are empty, so without any additional parameters the program always uses the default decoding. LANG LC_ALL LC_CTYPE Define current locale settings. Used to determine current default charset (if your Python is properly installed and configured). The program may produce incorrect MIME message. The purpose of the program is to decode whatever it is possible to decode, not to produce absolutely correct MIME output. The incorrect parts are obvious - decoded From/To/Cc/Reply-To/Mail-Followup-To/Subject headers and filenames. Other than that output is correct MIME message. The program does not try to guess whether the headers are correct. For example, if a message header states that charset is iso8859-5, but the body is actually in utf-8 the program will recode the message with the wrong charset. Oleg Broytman phd@phdru.name Copyright (C) 2001-2014 PhiloSoft Design. GNU GPL This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. mimedecode.py home page: http://phdru.name/Software/Python/#mimedecode