1 <?xml version="1.0" standalone="no"?>
2 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
3 "file:///usr/share/xml/docbook/schema/dtd/4.5/docbookx.dtd">
5 <refentry id="mimedecode.py">
8 <title>mimedecode.py</title>
9 <productname>mimedecode.docbook</productname>
11 <firstname>Oleg</firstname>
12 <surname>Broytman</surname>
13 <email>phd@phdru.name</email>
17 <year>2001-2014</year>
18 <holder>PhiloSoft Design.</holder>
23 <refentrytitle>mimedecode.py</refentrytitle>
24 <manvolnum>1</manvolnum>
28 <refname>mimedecode.py</refname>
29 <refpurpose>decode MIME message</refpurpose>
34 <command>mimedecode.py</command>
36 <option>-h|--help</option>
39 <option>-V|--version</option>
42 <option>-cCDP</option>
45 <option>-f charset</option>
48 <option>-H|--host=hostname</option>
51 <option>-d header1[,header2,header3...]</option>
54 <option>-d *[,-header1,-header2,-header3...]</option>
57 <option>-p header1[,header2,header3,...]:param1[,param2,param3,...]</option>
60 <option>-p *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</option>
63 <option>-p header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</option>
66 <option>-p *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</option>
69 <option>-r header1[,header2,header3...]</option>
72 <option>-r *[,-header1,-header2,-header3...]</option>
75 <option>-R header1[,header2,header3,...]:param1[,param2,param3,...]</option>
78 <option>-R *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</option>
81 <option>-R header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</option>
84 <option>-R *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</option>
87 <option>--set-header header:value</option>
90 <option>--set-param header:param=value</option>
93 <option>-BbeIit mask</option>
96 <option>--save-headers|body|message mask</option>
99 <option>-O dest_dir</option>
102 <option>-o output_file</option>
104 <arg choice="opt">input_file
105 <arg choice="opt">output_file</arg>
112 <title>DESCRIPTION</title>
114 Mail users, especially in non-English countries, often find that mail
115 messages arrived in different formats, with different content types, in
116 different encodings and charsets. Usually it is good because it allows to
117 use an appropriate format/encoding/whatever. Sometimes, though, some
118 unification is desirable. For example, one may want to put mail messages
119 into an archive, make HTML indices, run search indexer, etc. In such
120 situations converting messages to text in one character set and skipping
121 some binary attachments is much desirable.
125 Here is a solution - mimedecode.py!
129 This is a program to decode MIME messages. The program expects one input
130 file (either on command line or on stdin) which is treated as an RFC822
131 message, and decodes to stdout or an output file. If the file is not an
132 RFC822 message it is just copied to the output one-to-one. If the file is a
133 simple RFC822 message it is decoded as one part. If it is a MIME message
134 with multiple parts ("attachments") all non-multipart subparts are decoded.
135 Decoding can be controlled by the command-line options.
139 First, for every part the program removes headers and parameters listed with
140 -r and -R options. Then, Subject and Content-Disposition headers (and all
141 headers listed with -d and -p options) are examined. If any of those exists,
142 they are decoded according to RFC2047. Content-Disposition header is not
143 decoded (if it was not listed in option -d) - only its "filename" parameter.
144 Encoded header parameters violate the RFC, but widely deployed anyway by
145 ignorant coders who never even heard about RFCs. Correct parameter encoding
146 specified by RFC2231. This program decodes RFC2231-encoded parameters, too.
150 Then the body of the message (or the current part) is decoded. Decoding
151 starts with looking at header Content-Transfer-Encoding. If the header
152 specifies non-8bit encoding (usually base64 or quoted-printable), the body
153 is converted to 8bit (can be prevented with -B). Then if its content type
154 is multipart (multipart/related or multipart/mixed, e.g) every part is
155 recursively decoded. If it is not multipart, mailcap database is consulted
156 to find a way to convert the body to plain text (can be prevented with
157 options -Bbei). (The author has no idea how mailcap can be configured on
158 OSes other than POSIX, please don't ask; users can consult an example at
159 <ulink url="http://phdru.name/Software/dotfiles/mailcap.html">http://phdru.name/Software/dotfiles/mailcap.html</ulink>).
160 The decoding process uses the first copiousoutput filter it can find. If
161 there are no filters the body just passed as is.
165 Then Content-Type header is consulted for charset. If it is not equal to the
166 current locale charset and recoding is allowed (see options -Cc) the body
167 text is recoded. Finally message headers and the body are flushed to stdout.
173 Please be reminded that in the following options asterisk is a shell
174 metacharacter and should be escaped or quoted. Either write -d \*,-h1,-h2
175 or -d '*,-h1,-h2' or such.
180 <title>OPTIONS</title>
187 Print brief usage help and exit.
194 <term>--version</term>
197 Print version and exit.
206 Recode different character sets in message bodies to the current
207 default charset; this is the default.
216 Do not recode character sets in message bodies.
222 <term>-f charset</term>
225 Force this charset to be used for recoding instead of charset from
232 <term>-H hostname</term>
233 <term>--host=hostname</term>
236 Use this hostname in X-MIME-Autoconverted headers instead of the
243 <term>-d header1[,header2,header3...]</term>
246 Add the header(s) to a list of headers to decode; initially the
247 list contains headers "From", "To", "Cc", "Reply-To",
248 "Mail-Followup-To" and "Subject".
254 <term>-d *[,-header1,-header2,-header3...]</term>
257 This variant completely changes headers decoding. First, the list of
258 headers to decode is cleared (as with -D). Then all the headers are
259 decoded except the given list of exceptions (headers listed with
260 '-'). In this mode it would be meaningless to give more than one -d
261 options but the program doesn't enforce the limitation.
270 Clear the list of headers to decode (make it empty).
276 <term>-p header1[,header2,header3,...]:param1[,param2,param3,...]</term>
279 Add the parameter(s) to a list of headers parameters to decode;
280 the parameter(s) will be decoded only for the given header(s).
281 Initially the list contains header "Content-Type", parameter "name";
282 and header "Content-Disposition", parameter "filename".
288 <term>-p *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</term>
291 Add the parameter(s) to a list of headers parameters to decode;
292 the parameter(s) will be decoded for all headers except the given
299 <term>-p header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</term>
302 Decode all parameters except listed for the given list of headers.
308 <term>-p *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</term>
311 Decode all parameters except listed for all headers (except listed).
320 Clear the list of headers parameters to decode (make it empty).
326 <term>-r header1[,header2,header3...]</term>
329 Add the header(s) to a list of headers to remove completely;
330 initially the list is empty.
336 <term>-r *[,-header1,-header2,-header3...]</term>
339 Remove all headers except listed.
345 <term>-R header1[,header2,header3,...]:param1[,param2,param3,...]</term>
348 Add the parameter(s) to a list of headers parameters to remove;
349 the parameter(s) will be removed only for the given header(s).
350 Initially the list is empty.
356 <term>-R *[,-header1,-header2,-header3,...]:param1[,param2,param3,...]</term>
360 <term>-R header1[,header2,header3,...]:*[,-param1,-param2,-param3,...]</term>
364 <term>-R *[,-header1,-header2,-header3,...]:*[,-param1,-param2,-param3,...]</term>
367 Remove listed parameters (or all parameters except listed) from
368 these headers (or from all headers except listed).
374 <term>--set-header header:value</term>
377 The program sets or changes value for the header to the given value
378 (only at the top-level message).
384 <term>--set-param header:param=value</term>
387 The program sets or changes value for the header's parameter to the
388 given value (only at the top-level message). The header must exist.
397 Append mask to the list of binary content types that will be not
398 content-transfer-decoded (will be left as base64 or such).
407 Append mask to the list of binary content types; if the message to
408 decode has a part of this type the program content-transfer-decodes
409 (base64 or whatever to 8bit binary) it and outputs the decoded part
410 as is, without any further processing.
419 Append mask to the list of error content types; if the message to
420 decode has a part of this type the program fails with ValueError.
429 Append mask to the list of content types to completely ignore.
430 There will be no output - no headers, no body, no warning. For a
431 multipart part the entire subtree is removed.
440 Append mask to the list of content types to ignore; if the message
441 to decode has a part of this type the program outputs headers but
442 skips the body. Instead a line "Message body of type %s skipped."
452 Append mask to the list of content types to convert to text; if the
453 message to decode has a part of this type the program consults
454 mailcap database, find the first copiousoutput filter and, if any
455 filter is found, converts the part.
461 <term>--save-headers mask</term>
465 <term>--save-body mask</term>
469 <term>--save-message mask</term>
472 Append mask to lists of content types to save to files;
473 --save-headers saves only decoded headers of the message (or
474 the current subpart); --save-body saves only decoded body;
475 --save-message saves the entire message or subpart (headers +
482 <term>-O dest_dir</term>
485 Set destination directory for the output files; if the directory
486 doesn't exist it will be created. Default is the current directory.
492 <term>-o output_file</term>
495 Save output to the file related to the destination directory from
496 option -O. Also useful in case of redirected stdin:
497 <programlisting language="sh">mimedecode.py -o output_file < input_file
498 cat input_file | mimedecode.py -o output_file</programlisting>
505 The 5 list options (-Bbeit) require more explanation. They allow a user to
506 control body decoding with great flexibility. Think about said mail archive;
507 for example, its maintainer wants to put there only texts, convert
508 PDF/Postscript to text, pass HTML and images decoding base64 to html but
509 leaving images encoded, and ignore everything else. This is how it could be
515 mimedecode.py -t application/pdf -t application/postscript -t text/plain
516 -b text/html -B 'image/*' -i '*/*'
521 When the program decodes a message (non-MIME or a non-multipart subpart of a
522 MIME message), it consults Content-Type header. The content type is searched
523 in all 5 lists, in order "text-binary-ignore-error". If found, appropriate
524 action is performed. If not found, the program searches the same lists for
525 "type/*" mask (the type of "text/html" is just "text"). If found,
526 appropriate action is performed. If not found, the program searches the same
527 lists for "*/*" mask. If found, appropriate action is performed. If not
528 found, the program uses the default action, which is to decode everything to
529 text (if mailcap specifies a filter). This algorithm allows more specific
530 content types to override less specific: -b image/* will be processed
535 Options -e/-I/-i can also work with multipart subparts of a MIME message. In
536 case of -I/-i the entire subtree of that multipart is removed; with -i it's
537 replaced with ignore warning.
541 Initially all 5 lists are empty, so without any additional parameters
542 the program always uses the default decoding (as -t */*).
546 The 3 save options (--save-headers/body/message) are similar. They make the
547 program to save every non-multipart subpart (only headers, or body, or the
548 entire subpart: headers + body) that corresponds to the given mask to a file.
549 Before saving the message (or the subpart) is decoded according to all other
550 options and is placed to the output stream as usual. Filename for the file is
551 created using "filename" parameter from the Content-Disposition header, or
552 "name" parameter from the Content-Type header if one of those exist; a serial
553 counter is prepended to the filename to avoid collisions; if there are no
554 name/filename parameters, or the name/filename parameters contain forbidden
555 characters (null, slash, backslash) the filename is just the serial counter.
559 If the file doesn't have any extensions (no dots in the value of the
560 name/filename parameters, or the name is just the counter) the program tries
561 to guess an extension by looking up the content type in mime.types files
562 including .mime.types file in the user's home directory (if it exists). If
563 the file has an extension the program doesn't try to verify that it
564 corresponds to the content type.
568 The file is saved in the directory set with -O (default is the current
569 directory). The save options are proceeded before -e options so the user can
570 save the message that causes an error.
576 <title>ENVIRONMENT</title>
578 <varlistentry><term>LANG</term></varlistentry>
579 <varlistentry><term>LC_ALL</term></varlistentry>
580 <varlistentry><term>LC_CTYPE</term></varlistentry>
583 Define current locale settings. Used to determine current default charset (if
584 your Python is properly installed and configured).
592 The program may produce incorrect MIME message. The purpose of the program
593 is to decode whatever it is possible to decode, not to produce absolutely
594 correct MIME output. The incorrect parts are obvious - decoded
595 From/To/Cc/Reply-To/Mail-Followup-To/Subject headers and filenames. Other
596 than that output is correct MIME message. The program does not try to guess
597 whether the headers are correct. For example, if a message header states
598 that charset is iso8859-1, but the body (HTML, for example) is actually in
599 utf-8 the program will recode the message with the wrong charset.
605 <title>AUTHOR</title>
607 <firstname>Oleg</firstname>
608 <surname>Broytman</surname>
609 <email>phd@phdru.name</email>
615 <title>COPYRIGHT</title>
617 Copyright (C) 2001-2014 PhiloSoft Design.
623 <title>LICENSE</title>
631 <title>NO WARRANTIES</title>
633 This program is distributed in the hope that it will be useful, but WITHOUT
634 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
635 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
642 <title>SEE ALSO</title>
644 mimedecode.py home page:
645 <ulink url="http://phdru.name/Software/Python/#mimedecode">http://phdru.name/Software/Python/#mimedecode</ulink>