[cmucl-imp] run-program encoding

[Raymond Toy]

On 7/5/10 9:01 AM, Helmut Eller wrote:
> Is it possible to use utf-8 when RUN-PROGRAM is called like
>
> (run-program ... () :input :stream :output :stream :error :output :wait nil)
>
> ?
>   
I think so.  You will need to bind *default-external-format* to :utf-8
to get the input and output streams created with an external format of
:utf8.

I did a quick test with the following and it seems to work.

Note however, that I just discovered a critical bug in buffering for the
fast external format streams.  Depending on the encoding boundaries and
the internal buffer filling, you could get random replacement characters
instead of the correct characters.  I think this is fixed in CVS, but I
still need to do a bit more testing.

Until this is fixed, it's best to (setf lisp::*enable-stream-buffer-p*
nil).  This slows down decoding, but you won't get incorrect inputs.

Ray

echo.c:
#include <stdio.h>

int main()
{
  while (1) {
    int c;
    c = getchar();
    if (c == EOF) break;
    putchar(c);
    fflush(stdout);
  }
}

test-ec.lisp:
(defun test-ec ()
  (let* ((*default-external-format* :utf8)
     (p (run-program "ec" nil
             :input :stream :output :stream
             :wait nil)))
    (write-string (coerce '(#\a #\b #\u+391 #\u+392 #\u+1800) 'string)
(process-input p))
    (terpri (process-input p))
    (finish-output (process-input p))
    (print (read-line (process-output p)))
    (process-close p)))