CMUCL commit: src (47 files)
Raymond Toy
rtoy at common-lisp.net
Mon Jul 12 15:58:42 CEST 2010
Date: Monday, July 12, 2010 @ 09:58:42
Author: rtoy
Path: /project/cmucl/cvsroot/src
Added: bootfiles/20a/boot-2010-07-1.lisp
Modified: code/exports.lisp code/extfmts.lisp docs/cmu-user/unicode.tex
general-info/release-20b.txt
pcl/simple-streams/external-formats/ascii.lisp
pcl/simple-streams/external-formats/beta-gk.lisp
pcl/simple-streams/external-formats/cp1250.lisp
pcl/simple-streams/external-formats/cp1251.lisp
pcl/simple-streams/external-formats/cp1252.lisp
pcl/simple-streams/external-formats/cp1253.lisp
pcl/simple-streams/external-formats/cp1254.lisp
pcl/simple-streams/external-formats/cp1255.lisp
pcl/simple-streams/external-formats/cp1256.lisp
pcl/simple-streams/external-formats/cp1257.lisp
pcl/simple-streams/external-formats/cp1258.lisp
pcl/simple-streams/external-formats/cr.lisp
pcl/simple-streams/external-formats/crlf.lisp
pcl/simple-streams/external-formats/final-sigma.lisp
pcl/simple-streams/external-formats/iso8859-1.lisp
pcl/simple-streams/external-formats/iso8859-10.lisp
pcl/simple-streams/external-formats/iso8859-13.lisp
pcl/simple-streams/external-formats/iso8859-14.lisp
pcl/simple-streams/external-formats/iso8859-15.lisp
pcl/simple-streams/external-formats/iso8859-2.lisp
pcl/simple-streams/external-formats/iso8859-3.lisp
pcl/simple-streams/external-formats/iso8859-4.lisp
pcl/simple-streams/external-formats/iso8859-5.lisp
pcl/simple-streams/external-formats/iso8859-6.lisp
pcl/simple-streams/external-formats/iso8859-7.lisp
pcl/simple-streams/external-formats/iso8859-8.lisp
pcl/simple-streams/external-formats/iso8859-9.lisp
pcl/simple-streams/external-formats/koi8-r.lisp
pcl/simple-streams/external-formats/mac-cyrillic.lisp
pcl/simple-streams/external-formats/mac-greek.lisp
pcl/simple-streams/external-formats/mac-icelandic.lisp
pcl/simple-streams/external-formats/mac-latin2.lisp
pcl/simple-streams/external-formats/mac-roman.lisp
pcl/simple-streams/external-formats/mac-turkish.lisp
pcl/simple-streams/external-formats/utf-16-be.lisp
pcl/simple-streams/external-formats/utf-16-le.lisp
pcl/simple-streams/external-formats/utf-16.lisp
pcl/simple-streams/external-formats/utf-32-be.lisp
pcl/simple-streams/external-formats/utf-32-le.lisp
pcl/simple-streams/external-formats/utf-32.lisp
pcl/simple-streams/external-formats/utf-8.lisp
pcl/simple-streams/external-formats/void.lisp
Add a documentation slot to external formats so that we can give a
little information about the format. Provide a means to get a list of
external formats and to display the documentation.
bootfiles/20a/boot-2010-07-1.lisp:
o Use this bootstrap file when doing a normal build.
code/exports.lisp:
o New functions:
- Add LIST-ALL-EXTERNAL-FORMATS to list all available external formats
and their corresponding aliases.
- Add DESCRIBE-EXTERNAL-FORMAT to print some information about the
given format.
o Add docuemntation slot to defstruct EXTERNAL-FORMAT.
o Change DEFINE-EXTERNAL-FORMAT macro. Adds :DOCUMENTATION keyword to
specify the documentation. Add :BASE keyword indicate that the
external format is based on another format. (Previously, this
wasn't needed, but is somewhat incompatible with adding a
documentation string.)
o Change DEFINE-COMPOSING-EXTERNAL-FORMAT to include :documentation
keyword to specify the documentation for the format.
o Minor reindentation of some docstrings.
o Make sure documentation strings for external format are marked for
translation; wrap other strings with intl:gettext to explicitly mark
them for translations.
o Add docstring for VOID and ISO8859-1 external formats.
code/exports.lisp:
o Export the new symbols LIST-ALL-EXTERNAL-FORMATS and
DESCRIBE-EXTERNAL-FORMAT. Import into EXTENSIONS package.
docs/cmu-user/unicode.tex:
o Update docs to include LIST-ALL-EXTERNAL-FORMATS and
DESCRIBE-EXTERNAL-FORMAT.
o Update docs for DEFINE-EXTERNAL-FORMAT and
DEFINE-COMPOSING-EXTERNAL-FORMAT to match implementation.
general-info/release-20b.txt:
o Update
external-formats/*.lisp:
o Update with docstrings.
o Add :BASE keyword where needed.
--------------------------------------------------------+
bootfiles/20a/boot-2010-07-1.lisp | 34 +++
code/exports.lisp | 14 -
code/extfmts.lisp | 164 ++++++++++-----
docs/cmu-user/unicode.tex | 59 +++--
general-info/release-20b.txt | 4
pcl/simple-streams/external-formats/ascii.lisp | 7
pcl/simple-streams/external-formats/beta-gk.lisp | 7
pcl/simple-streams/external-formats/cp1250.lisp | 10
pcl/simple-streams/external-formats/cp1251.lisp | 10
pcl/simple-streams/external-formats/cp1252.lisp | 8
pcl/simple-streams/external-formats/cp1253.lisp | 8
pcl/simple-streams/external-formats/cp1254.lisp | 8
pcl/simple-streams/external-formats/cp1255.lisp | 8
pcl/simple-streams/external-formats/cp1256.lisp | 8
pcl/simple-streams/external-formats/cp1257.lisp | 8
pcl/simple-streams/external-formats/cp1258.lisp | 8
pcl/simple-streams/external-formats/cr.lisp | 7
pcl/simple-streams/external-formats/crlf.lisp | 7
pcl/simple-streams/external-formats/final-sigma.lisp | 6
pcl/simple-streams/external-formats/iso8859-1.lisp | 20 +
pcl/simple-streams/external-formats/iso8859-10.lisp | 9
pcl/simple-streams/external-formats/iso8859-13.lisp | 8
pcl/simple-streams/external-formats/iso8859-14.lisp | 9
pcl/simple-streams/external-formats/iso8859-15.lisp | 10
pcl/simple-streams/external-formats/iso8859-2.lisp | 11 -
pcl/simple-streams/external-formats/iso8859-3.lisp | 10
pcl/simple-streams/external-formats/iso8859-4.lisp | 9
pcl/simple-streams/external-formats/iso8859-5.lisp | 10
pcl/simple-streams/external-formats/iso8859-6.lisp | 9
pcl/simple-streams/external-formats/iso8859-7.lisp | 9
pcl/simple-streams/external-formats/iso8859-8.lisp | 8
pcl/simple-streams/external-formats/iso8859-9.lisp | 8
pcl/simple-streams/external-formats/koi8-r.lisp | 8
pcl/simple-streams/external-formats/mac-cyrillic.lisp | 9
pcl/simple-streams/external-formats/mac-greek.lisp | 9
pcl/simple-streams/external-formats/mac-icelandic.lisp | 9
pcl/simple-streams/external-formats/mac-latin2.lisp | 10
pcl/simple-streams/external-formats/mac-roman.lisp | 9
pcl/simple-streams/external-formats/mac-turkish.lisp | 10
pcl/simple-streams/external-formats/utf-16-be.lisp | 11 -
pcl/simple-streams/external-formats/utf-16-le.lisp | 11 -
pcl/simple-streams/external-formats/utf-16.lisp | 12 -
pcl/simple-streams/external-formats/utf-32-be.lisp | 12 -
pcl/simple-streams/external-formats/utf-32-le.lisp | 12 -
pcl/simple-streams/external-formats/utf-32.lisp | 12 -
pcl/simple-streams/external-formats/utf-8.lisp | 8
pcl/simple-streams/external-formats/void.lisp | 5
47 files changed, 501 insertions(+), 161 deletions(-)
Index: src/bootfiles/20a/boot-2010-07-1.lisp
diff -u /dev/null src/bootfiles/20a/boot-2010-07-1.lisp:1.1
--- /dev/null Mon Jul 12 09:58:43 2010
+++ src/bootfiles/20a/boot-2010-07-1.lisp Mon Jul 12 09:58:42 2010
@@ -0,0 +1,34 @@
+;; Simple bootstrap file to add the documentation slot to the EFX and
+;; EXTERNAL-FORMAT structure.
+
+(in-package "STREAM")
+
+(ext:without-package-locks
+ (handler-bind ((error (lambda (c)
+ (declare (ignore c))
+ (invoke-restart 'kernel::clobber-it))))
+ #+nil
+ (defstruct efx
+ (octets-to-code #'%efni :type function :read-only t)
+ (code-to-octets #'%efni :type function :read-only t)
+ (flush-state nil :type (or null function) :read-only t)
+ (copy-state nil :type (or null function) :read-only t)
+ (cache nil :type (or null simple-vector))
+ (min 1 :type kernel:index :read-only t)
+ (max 1 :type kernel:index :read-only t)
+ (documentation nil :type (or null string) :read-only t)))
+
+ (handler-bind ((error (lambda (c)
+ (declare (ignore c))
+ (invoke-restart 'kernel::clobber-it))))
+ (defstruct (external-format
+ (:conc-name ef-)
+ (:print-function %print-external-format)
+ (:constructor make-external-format (name efx composingp documentation
+ &optional slots slotd)))
+ (name (ext:required-argument) :type (or keyword cons) :read-only t)
+ (efx (ext:required-argument) :type efx :read-only t)
+ (composingp (ext:required-argument) :type boolean :read-only t)
+ (slots #() :type simple-vector :read-only t)
+ (slotd nil :type list :read-only t)
+ (documentation nil :type (or null string) :read-only t))))
\ No newline at end of file
Index: src/code/exports.lisp
diff -u src/code/exports.lisp:1.297 src/code/exports.lisp:1.298
--- src/code/exports.lisp:1.297 Sat Jul 10 18:50:58 2010
+++ src/code/exports.lisp Mon Jul 12 09:58:42 2010
@@ -5,7 +5,7 @@
;;; Carnegie Mellon University, and has been placed in the public domain.
;;;
(ext:file-comment
- "$Header: /project/cmucl/cvsroot/src/code/exports.lisp,v 1.297 2010-07-10 22:50:58 rtoy Exp $")
+ "$Header: /project/cmucl/cvsroot/src/code/exports.lisp,v 1.298 2010-07-12 13:58:42 rtoy Exp $")
;;;
;;; **********************************************************************
;;;
@@ -1251,9 +1251,9 @@
;; Make these STREAM symbols available
(dolist
(name
- '("STRING-TO-OCTETS" "OCTETS-TO-STRING" "*DEFAULT-EXTERNAL-FORMAT*"
- "STRING-ENCODE" "STRING-DECODE" "SET-SYSTEM-EXTERNAL-FORMAT"
- "LIST-ALL-EXTERNAL-FORMATS"))
+ '("STRING-TO-OCTETS" "OCTETS-TO-STRING" "*DEFAULT-EXTERNAL-FORMAT*"
+ "STRING-ENCODE" "STRING-DECODE" "SET-SYSTEM-EXTERNAL-FORMAT"
+ "LIST-ALL-EXTERNAL-FORMATS" "DESCRIBE-EXTERNAL-FORMAT"))
(intern name "STREAM"))
(defpackage "EXTENSIONS"
@@ -1484,7 +1484,8 @@
"STRING-TO-OCTETS" "OCTETS-TO-STRING" "*DEFAULT-EXTERNAL-FORMAT*"
"STRING-ENCODE" "STRING-DECODE"
"SET-SYSTEM-EXTERNAL-FORMAT"
- "LIST-ALL-EXTERNAL-FORMATS")
+ "LIST-ALL-EXTERNAL-FORMATS"
+ "DESCRIBE-EXTERNAL-FORMAT")
;; Unicode
(:export "STRING-TO-OCTETS" "OCTETS-TO-STRING" "*DEFAULT-EXTERNAL-FORMAT*"
"LIST-ALL-EXTERNAL-FORMATS"
@@ -1555,7 +1556,8 @@
"STRING-ENCODE" "STRING-DECODE"
"SET-SYSTEM-EXTERNAL-FORMAT"
"+REPLACEMENT-CHARACTER-CODE+"
- "LIST-ALL-EXTERNAL-FORMATS"))
+ "LIST-ALL-EXTERNAL-FORMATS"
+ "DESCRIBE-EXTERNAL-FORMAT"))
(defpackage "LOOP")
(dolist
Index: src/code/extfmts.lisp
diff -u src/code/extfmts.lisp:1.34 src/code/extfmts.lisp:1.35
--- src/code/extfmts.lisp:1.34 Sat Jul 10 18:50:58 2010
+++ src/code/extfmts.lisp Mon Jul 12 09:58:42 2010
@@ -5,7 +5,7 @@
;;; domain.
;;;
(ext:file-comment
- "$Header: /project/cmucl/cvsroot/src/code/extfmts.lisp,v 1.34 2010-07-10 22:50:58 rtoy Exp $")
+ "$Header: /project/cmucl/cvsroot/src/code/extfmts.lisp,v 1.35 2010-07-12 13:58:42 rtoy Exp $")
;;;
;;; **********************************************************************
;;;
@@ -18,7 +18,8 @@
(export '(string-to-octets octets-to-string *default-external-format*
string-encode string-decode set-system-external-format
+replacement-character-code+
- list-all-external-formats))
+ list-all-external-formats
+ describe-external-format))
(defvar *default-external-format*
:iso8859-1
@@ -58,6 +59,14 @@
(declare (ignore condition))
(format stream (intl:gettext "Attempting unimplemented external-format I/O.")))))
+(define-condition external-format-not-found (error)
+ ((name :reader external-format-not-found-name
+ :initarg :name))
+ (:report
+ (lambda (condition stream)
+ (format stream (intl:gettext "External format ~S not found.")
+ (external-format-not-found-name condition)))))
+
(defun %efni (a b c d)
(declare (ignore a b c d))
(error 'external-format-not-implemented))
@@ -86,18 +95,22 @@
(min 1 :type kernel:index :read-only t)
;;
;; Maximum number of octets needed to form a codepoint.
- (max 1 :type kernel:index :read-only t))
+ (max 1 :type kernel:index :read-only t)
+ ;;
+ ;; Documentation for this external format
+ #+nil(documentation nil :type (or null string) :read-only t))
(defstruct (external-format
(:conc-name ef-)
(:print-function %print-external-format)
- (:constructor make-external-format (name efx composingp
+ (:constructor make-external-format (name efx composingp documentation
&optional slots slotd)))
(name (ext:required-argument) :type (or keyword cons) :read-only t)
(efx (ext:required-argument) :type efx :read-only t)
(composingp (ext:required-argument) :type boolean :read-only t)
(slots #() :type simple-vector :read-only t)
- (slotd nil :type list :read-only t))
+ (slotd nil :type list :read-only t)
+ (documentation nil :type (or null string) :read-only t))
(defun %print-external-format (ef stream depth)
(declare (ignore depth))
@@ -147,17 +160,21 @@
;;; DEFINE-EXTERNAL-FORMAT -- Public
;;;
-;;; name (&key min max size) (&rest slots) octets-to-code code-to-octets
-;;; flush-state copy-state
+;;; name (&key base min max size documentation) (&rest slots) octets-to-code
+;;; code-to-octets flush-state copy-state
+;;;
+;;; Define a new external format. If base is specified, then an
+;;; external format is defined that is based on a previously defined
+;;; external format named Base. The slot names used in Slots must
+;;; match those defined in the Base format.
+;;;
+;;; If Base is not specified, a new external format is defined.
+;;; Min/Max/Size are the minimum and maximum number of octets that
+;;; make up a character (:size N is just shorthand for :min N :max
+;;; N). Slots is a list of slot descriptions similar to defstruct.
;;;
-;;; Define a new external format. Min/Max/Size are the minimum and
-;;; maximum number of octets that make up a character (:size N is just
-;;; shorthand for :min N :max N). Slots is a list of slot descriptions
-;;; similar to defstruct.
-;;;
-;;; name (base) (&rest slots)
-;;; Define an external format based on a previously-defined external
-;;; format, Base. The slot names used in Slots must match those in Base.
+;;; In both cases, Documentation is a string that documents the
+;;; external format.
;;;
;;; octets-to-code (state input unput error &rest vars)
;;; Defines a form to be used by the external format to convert
@@ -213,28 +230,24 @@
;;; process characters outside the Lisp range (see
;;; CODEPOINT-TO-OCTETS, OCTETS-TO-CODEPOINT)
;;;
-(defmacro define-external-format (name (&rest args) (&rest slots)
+(defmacro define-external-format (name (&key base min max size (documentation ""))
+ (&rest slots)
&optional octets-to-code code-to-octets
flush-state copy-state)
- (when (and (oddp (length args)) (not (= (length args) 1)))
- (warn (intl:gettext "Nonsensical argument (~S) to DEFINE-EXTERNAL-FORMAT.") args))
(let* ((tmp (gensym))
- (min (if (evenp (length args))
- (or (getf args :min) (getf args :size) 1)
- 1))
- (max (if (evenp (length args))
- (or (getf args :max) (getf args :size) 6)
- 6))
- (base (if (= (length args) 1)
- (find-external-format (first args))
- nil))
+ (min (or min size 1))
+ (max (or max size 6))
+ (base (when base
+ (find-external-format base)))
(bslotd (if base (ef-slotd base) nil))
(slotd (%merge-slots bslotd slots))
(slotb (loop for slot in slotd
collect `(,(first slot)
`(the ,',(fourth slot)
;; IDENTITY is here to protect against SETF
- (identity (svref %slots% ,',(second slot))))))))
+ (identity (svref %slots% ,',(second slot))))))))
+ (when documentation
+ (intl::note-translatable intl::*default-domain* documentation))
`(macrolet ((octets-to-code ((state input unput error &rest vars) body)
`(lambda (,state ,input ,unput ,error)
(declare (ignorable ,state ,input ,unput ,error)
@@ -274,8 +287,10 @@
:copy-state ,copy-state
:cache (make-array +ef-max+
:initial-element nil)
- :min ,(min min max) :max ,(max min max)))
+ :min ,(min min max)
+ :max ,(max min max)))
nil
+ ,documentation
(let* ,(loop for x in slotd
collect (list (first x) (third x)))
(vector ,@(mapcar #'first slotd)))
@@ -290,11 +305,11 @@
;;; to be of any use.
;;;
;;;
-;;; name (&key min max size) input output
+;;; name (&key min max size documentation) input output
;;; Defines a new composing external format. The parameters Min,
-;;; Max, and Size are the same as for defining an external format.
-;;; The parameters input and output are forms to handle input and
-;;; output.
+;;; Max, Size, and Documentation are the same as for defining an
+;;; external format. The parameters input and output are forms to
+;;; handle input and output.
;;;
;;; input (state input unput &rest vars)
;;; Defines a form to be used by the composing external format when
@@ -319,7 +334,7 @@
;;; value of the stream's state variable. Output is a form that
;;; writes one octet to the output stream.
-(defmacro define-composing-external-format (name (&key min max size)
+(defmacro define-composing-external-format (name (&key min max size documentation)
input output)
(let ((tmp (gensym))
(min (or min size 1))
@@ -347,6 +362,7 @@
:code-to-octets ,output
:min ,(min min max) :max ,(max min max))
t
+ ,documentation
#() '())))))
(defun load-external-format-aliases ()
@@ -395,6 +411,49 @@
ef)
(sort result #'string< :key #'first)))
+(defun describe-external-format (external-format)
+ "Print a description of the given External-Format. This may cause
+ the external format to be loaded (silently), if it is not already
+ loaded."
+ (when (zerop (hash-table-count
+ *external-format-aliases*))
+ (load-external-format-aliases))
+ (let ((alias (gethash external-format *external-format-aliases*)))
+ (cond (alias
+ (format t (intl:gettext "~&~S is an alias for the external format ~S.~2%")
+ external-format alias))
+ ((and (listp external-format)
+ (> (length external-format) 1))
+ ;; Some kind of composed external format
+ (format t (intl:gettext "~&~S is a composed external format.~2%") external-format))
+ (t
+ (let ((ef (handler-case (let ((*compile-print* nil)
+ (ext:*compile-progress* nil)
+ (*compile-verbose* nil))
+ ;; Should we be this silent when
+ ;; loading the external format?
+ ;; We aren't when the normally
+ ;; loading the format.
+ (find-external-format external-format))
+ (external-format-not-found ()
+ (format *error-output*
+ (intl:gettext "~&Could not find external format ~S~%")
+ external-format)))))
+ (when ef
+ (let (aliases)
+ ;; Find any aliases for this external format. Doesn't need to be efficient.
+ (maphash #'(lambda (k v)
+ (when (eq v external-format)
+ (push k aliases)))
+ *external-format-aliases*)
+ (format t (intl:gettext "~S~:[~; - [Aliases: ~{~S~^, ~}~]]~%")
+ external-format aliases aliases))
+ (when (ef-composingp ef)
+ (format t (intl:gettext "~&~S is a composing external format.~2%")
+ external-format))
+ (format t "~&~A~%"
+ (intl:gettext (or (ef-documentation ef) "")))))))))
+
(defun %find-external-format (name)
;; avoid loading files, etc., early in the boot sequence
(when (or (eq name :iso8859-1)
@@ -461,6 +520,7 @@
:cache (make-array +ef-max+ :initial-element nil)
:min (* (ef-min-octets a) (ef-min-octets b))
:max (* (ef-max-octets a) (ef-max-octets b)))
+ nil
nil #() '()))
(defun find-external-format (name &optional (error-p t))
@@ -479,7 +539,7 @@
(flet ((not-found ()
(when (equal *default-external-format* name)
(setq *default-external-format* :iso8859-1))
- (if error-p (error (intl:gettext "External format ~S not found.") name) nil)))
+ (if error-p (error 'external-format-not-found :name name) nil)))
(if (consp name)
(let ((efs (mapcar #'%find-external-format name)))
(if (member nil efs)
@@ -568,19 +628,29 @@
(declare (ignore condition))
(format stream (intl:gettext "Attempting I/O through void external-format.")))))
-(define-external-format :void (:size 0) ()
+(define-external-format :void (:size 0 :documentation
+"Void external format that signals an error on any input or output.")
+ ()
(octets-to-code (state input unput error)
`(error 'void-external-format))
(code-to-octets (code state output error)
`(error 'void-external-format)))
-(define-external-format :iso8859-1 (:size 1) ()
+(define-external-format :iso8859-1 (:size 1 :documentation
+"ISO8859-1 is an 8-bit character encoding generally intended for
+Western European languages including English, German, Italian,
+Norwegian, Portuguese, Spanish, Swedish and many others.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
+ ()
(octets-to-code (state input unput error)
`(values ,input 1))
(code-to-octets (code state output error)
`(,output (if (> ,code 255)
(if ,error
- (funcall ,error "Cannot output codepoint #x~X to ISO8859-1 stream"
+ (funcall ,error
+ (intl:gettext "Cannot output codepoint #x~X to ISO8859-1 stream")
,code 1)
#x3F)
,code))))
@@ -684,10 +754,10 @@
(if ,error
(if (lisp::surrogatep code)
(funcall ,error
- ,(format nil "Surrogate codepoint #x~~4,'0X is illegal for ~A"
- external-format)
+ (format nil (intl:gettext "Surrogate codepoint #x~~4,'0X is illegal for ~A")
+ ,external-format)
code nil)
- (funcall ,error "Illegal codepoint on input: #x~X" code nil))
+ (funcall ,error (intl:gettext "Illegal codepoint on input: #x~X") code nil))
#-(and unicode (not unicode-bootstrap)) #\?
#+(and unicode (not unicode-bootstrap)) #\U+FFFD))
#+unicode
@@ -718,7 +788,8 @@
(,wryte (if (lisp::surrogatep (char-code ,nchar) :low)
(surrogates-to-codepoint (car ,nstate) ,nchar)
(if ,error
- (funcall ,error "Cannot convert invalid surrogate #x~X to character"
+ (funcall ,error
+ (intl:gettext "Cannot convert invalid surrogate #x~X to character")
,nchar)
+replacement-character-code+)))
(setf (car ,nstate) nil))
@@ -726,7 +797,8 @@
;; the replacement character.
(,wryte (if (lisp::surrogatep (char-code ,nchar) :low)
(if ,error
- (funcall ,error "Cannot convert lone trailing surrogate #x~X to character"
+ (funcall ,error
+ (intl:gettext "Cannot convert lone trailing surrogate #x~X to character")
,nchar)
+replacement-character-code+)
(char-code ,nchar)))))))))
@@ -763,9 +835,9 @@
(buffer nil bufferp)
error)
"Convert String to octets using the specified External-format. The
- string is bounded by Start (defaulting to 0) and End (defaulting to
- the end of the string. If Buffer is given, the octets are stored
- there. If not, a new buffer is created."
+ string is bounded by Start (defaulting to 0) and End (defaulting to
+ the end of the string. If Buffer is given, the octets are stored
+ there. If not, a new buffer is created."
(declare (type string string)
(type kernel:index start)
(type (or kernel:index null) end)
Index: src/docs/cmu-user/unicode.tex
diff -u src/docs/cmu-user/unicode.tex:1.10 src/docs/cmu-user/unicode.tex:1.11
--- src/docs/cmu-user/unicode.tex:1.10 Sat Jul 3 17:41:21 2010
+++ src/docs/cmu-user/unicode.tex Mon Jul 12 09:58:42 2010
@@ -549,6 +549,18 @@
converting the octets.
\end{defun}
+\begin{defun}{}{list-all-external-formats}{}
+ \code{list-all-external-formats} produces a list of all known
+ external formats and the known aliases for for them. Each element
+ of the list is a list consisting of the name of the external format
+ and a list of the known aliases for the format.
+\end{defun}
+
+\begin{defun}{}{describe-external-format}{\args \var{external-format}}
+ \code{describe-external-format} prints a description of the
+ specified external-format.
+\end{defun}
+
\section{Writing External Formats}
\subsection{External Formats}
@@ -562,31 +574,34 @@
\code{stream::define-external-format}.
\begin{defmac}[base]{stream:}{define-external-format}{\args \var{name}
- (\keys{\var{min} \var{max} \var{size}}) (\amprest{} \var{slots})
+ (\keys{\var{base} \var{min} \var{max} \var{size}
+ \var{documentation}})
+ (\amprest{} \var{slots})
\morekeys{\var{octets-to-code} \var{code-to-octets}
\var{flush-state} \var{copy-state}}}
- \defmacx[stream:]{define-external-format}{\args \var{name}
- (\var{base}) (\amprest{} \var{slots})}
- The first defines a new external format of the name \kwd{name}.
- \var{min}, \var{max}, and \var{size} are the minimum and maximum
- number of octets that make up a character. (\code{\kwd{size} n} is
- just a short cut for \code{\kwd{min} n \kwd{max} n}.) The arguments
- \var{octets-to-code} and \var{code-to-octets} are not optional in
- this case. They specify how to convert octets to codepoints and
- vice versa, respectively. These should be backquoted forms for the
- body of a function to do the conversion. See the description below
- for these functions. Some good examples are the external format for
+
+ If \kwd{base} is not given, this defines a new external format of
+ the name \kwd{name}. \var{min}, \var{max}, and \var{size} are the
+ minimum and maximum number of octets that make up a character.
+ (\code{\kwd{size} n} is just a short cut for \code{\kwd{min} n
+ \kwd{max} n}.) The description of the external format can be
+ given using \kwd{documentation}. The arguments \var{octets-to-code}
+ and \var{code-to-octets} are not optional in this case. They
+ specify how to convert octets to codepoints and vice versa,
+ respectively. These should be backquoted forms for the body of a
+ function to do the conversion. See the description below for these
+ functions. Some good examples are the external format for
\kwd{utf-8} or \kwd{utf-16}. The \kwd{slots} argument is a list of
- read-only slots, similar to defstruct. The slot names are available as
- local variables inside the \var{code-to-octets} and \var{octets-to-code}
- bodies.
-
- The second form above defines an external format with the name
- \kwd{name} that is based on a previously defined format \kwd{base}.
- The \var{slots} are inherited from the \kwd{base} format by default,
- although the definition may alter their values and add new slots.
- See, for example, the \kwd{mac-greek} external format.
+ read-only slots, similar to defstruct. The slot names are available
+ as local variables inside the \var{code-to-octets} and
+ \var{octets-to-code} bodies.
+
+ If \kwd{base} is given, then an external format is defined with the
+ name \kwd{name} that is based on a previously defined format
+ \kwd{base}. The \var{slots} are inherited from the \kwd{base} format
+ by default, although the definition may alter their values and add
+ new slots. See, for example, the \kwd{mac-greek} external format.
\end{defmac}
@@ -657,7 +672,7 @@
\subsection{Composing External Formats}
\begin{defmac}{stream:}{define-composing-external-format}{\args \var{name}
- (\keys{\var{min} \var{max} \var{size}}) \var{input}
+ (\keys{\var{min} \var{max} \var{size} \var{documentation}}) \var{input}
\var{output}}
This is the same as \code{define-external-format}, except that a
composing external format is created.
Index: src/general-info/release-20b.txt
diff -u src/general-info/release-20b.txt:1.35 src/general-info/release-20b.txt:1.36
--- src/general-info/release-20b.txt:1.35 Tue Jul 6 13:55:42 2010
+++ src/general-info/release-20b.txt Mon Jul 12 09:58:42 2010
@@ -75,6 +75,10 @@
function of 2 arguments: a message string and the offending
codepoint. If the function returns, it must be the codepoint of
the desired replacement.
+ - Add EXT:LIST-ALL-EXTERNAL-FORMATS to list all known external
+ formats and their aliases.
+ - ADD EXT:DESCRIBE-EXTERNAL-FORMAT to print a description of the
+ specified exernal format.
* ANSI compliance fixes:
- COMPILE will update the macro-function if the specified name
Index: src/pcl/simple-streams/external-formats/ascii.lisp
diff -u src/pcl/simple-streams/external-formats/ascii.lisp:1.4 src/pcl/simple-streams/external-formats/ascii.lisp:1.5
--- src/pcl/simple-streams/external-formats/ascii.lisp:1.4 Wed Jul 7 09:04:04 2010
+++ src/pcl/simple-streams/external-formats/ascii.lisp Mon Jul 12 09:58:42 2010
@@ -4,11 +4,14 @@
;;; This code was written by Raymond Toy and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/ascii.lisp,v 1.4 2010-07-07 13:04:04 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/ascii.lisp,v 1.5 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
-(define-external-format :ascii (:size 1)
+(define-external-format :ascii (:size 1 :documentation
+"US ASCII 7-bit encoding. Illegal input sequences are replaced with
+the Unicode replacment character. Illegal output characters are
+replaced with a question mark.")
()
(octets-to-code (state input unput error c)
`(let ((,c ,input))
Index: src/pcl/simple-streams/external-formats/beta-gk.lisp
diff -u src/pcl/simple-streams/external-formats/beta-gk.lisp:1.2 src/pcl/simple-streams/external-formats/beta-gk.lisp:1.3
--- src/pcl/simple-streams/external-formats/beta-gk.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/beta-gk.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/beta-gk.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/beta-gk.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
;; This is a composing format that translates (lower-case) Beta code
;; (an ASCII encoding of ancient Greek) into Unicode Greek.
@@ -190,7 +190,10 @@
#(42 47) #(42 40))))))))
-(define-composing-external-format :beta-gk (:min 1 :max 6) ; 6??
+(define-composing-external-format :beta-gk (:min 1 :max 6 ; 6??
+ :documentation
+"A composing external format that translates (lower-case) Beta code
+(an ASCII encoding of ancient Greek) into Unicode Greek.")
(input (state input unput ch wd ent nch nwd nent)
`(flet ((lookup (c map)
(loop while map do
Index: src/pcl/simple-streams/external-formats/cp1250.lisp
diff -u src/pcl/simple-streams/external-formats/cp1250.lisp:1.3 src/pcl/simple-streams/external-formats/cp1250.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1250.lisp:1.3 Sun Jun 21 11:12:23 2009
+++ src/pcl/simple-streams/external-formats/cp1250.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1250.lisp,v 1.3 2009-06-21 15:12:23 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1250.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -32,5 +32,11 @@
324 328 243 244 337 246 247 345 367 250
369 252 253 355 729)))
-(define-external-format :cp1250 (:mac-roman)
+(define-external-format :cp1250 (:base :mac-roman :documentation
+"CP1250 is a Windows code page to represent texts in Central and
+Eastern European languages such as Polish, Czech, Slovak, Hungarian,
+Slovene, Bosnian, Croation, Serbian, Romanian, and Albanian.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1250+)))
Index: src/pcl/simple-streams/external-formats/cp1251.lisp
diff -u src/pcl/simple-streams/external-formats/cp1251.lisp:1.3 src/pcl/simple-streams/external-formats/cp1251.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1251.lisp:1.3 Sun Jun 21 11:12:24 2009
+++ src/pcl/simple-streams/external-formats/cp1251.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1251.lisp,v 1.3 2009-06-21 15:12:24 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1251.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -30,5 +30,11 @@
1090 1091 1092 1093 1094 1095 1096 1097
1098 1099 1100 1101 1102 1103)))
-(define-external-format :cp1251 (:mac-roman)
+(define-external-format :cp1251 (:base :mac-roman :documentation
+"CP1251 is a Windows code page to represent texts that use the
+Cyrillic alphabet such as Russian, Bulgarian, Serbian Cyrillic, and
+others.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1251+)))
Index: src/pcl/simple-streams/external-formats/cp1252.lisp
diff -u src/pcl/simple-streams/external-formats/cp1252.lisp:1.4 src/pcl/simple-streams/external-formats/cp1252.lisp:1.5
--- src/pcl/simple-streams/external-formats/cp1252.lisp:1.4 Mon Aug 17 13:47:13 2009
+++ src/pcl/simple-streams/external-formats/cp1252.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1252.lisp,v 1.4 2009-08-17 17:47:13 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1252.lisp,v 1.5 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -28,5 +28,9 @@
241 242 243 244 245 246 247 248 249 250
251 252 253 254 255)))
-(define-external-format :cp1252 (:mac-roman)
+(define-external-format :cp1252 (:base :mac-roman :documentation
+"CP1252 is a Windows code page for the Latin alphabet.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1252+)))
Index: src/pcl/simple-streams/external-formats/cp1253.lisp
diff -u src/pcl/simple-streams/external-formats/cp1253.lisp:1.3 src/pcl/simple-streams/external-formats/cp1253.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1253.lisp:1.3 Sun Jun 21 10:03:09 2009
+++ src/pcl/simple-streams/external-formats/cp1253.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1253.lisp,v 1.3 2009-06-21 14:03:09 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1253.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -31,5 +31,9 @@
963 964 965 966 967 968 969 970 971 972
973 974 65534)))
-(define-external-format :cp1253 (:mac-roman)
+(define-external-format :cp1253 (:base :mac-roman :documentation
+"CP1253 is a Windows code page for Greek.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1253+)))
Index: src/pcl/simple-streams/external-formats/cp1254.lisp
diff -u src/pcl/simple-streams/external-formats/cp1254.lisp:1.3 src/pcl/simple-streams/external-formats/cp1254.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1254.lisp:1.3 Sun Jun 21 11:12:24 2009
+++ src/pcl/simple-streams/external-formats/cp1254.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1254.lisp,v 1.3 2009-06-21 15:12:24 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1254.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -30,5 +30,9 @@
287 241 242 243 244 245 246 247 248 249
250 251 252 305 351 255)))
-(define-external-format :cp1254 (:mac-roman)
+(define-external-format :cp1254 (:base :mac-roman :documentation
+"CP1254 is a Windows code page for Turkish.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1254+)))
Index: src/pcl/simple-streams/external-formats/cp1255.lisp
diff -u src/pcl/simple-streams/external-formats/cp1255.lisp:1.3 src/pcl/simple-streams/external-formats/cp1255.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1255.lisp:1.3 Sun Jun 21 11:12:24 2009
+++ src/pcl/simple-streams/external-formats/cp1255.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1255.lisp,v 1.3 2009-06-21 15:12:24 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1255.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -32,5 +32,9 @@
1505 1506 1507 1508 1509 1510 1511 1512
1513 1514 65534 65534 8206 8207 65534)))
-(define-external-format :cp1255 (:mac-roman)
+(define-external-format :cp1255 (:base :mac-roman :documentation
+"CP1255 is a Windows code page for Hebrew.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1255+)))
Index: src/pcl/simple-streams/external-formats/cp1256.lisp
diff -u src/pcl/simple-streams/external-formats/cp1256.lisp:1.3 src/pcl/simple-streams/external-formats/cp1256.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1256.lisp:1.3 Sun Jun 21 11:12:25 2009
+++ src/pcl/simple-streams/external-formats/cp1256.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1256.lisp,v 1.3 2009-06-21 15:12:25 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1256.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -32,5 +32,9 @@
247 1617 249 1618 251 252 8206 8207
1746)))
-(define-external-format :cp1256 (:mac-roman)
+(define-external-format :cp1256 (:base :mac-roman :documentation
+"CP1256 is a Windows code page for Arabic.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1256+)))
Index: src/pcl/simple-streams/external-formats/cp1257.lisp
diff -u src/pcl/simple-streams/external-formats/cp1257.lisp:1.3 src/pcl/simple-streams/external-formats/cp1257.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1257.lisp:1.3 Sun Jun 21 11:12:25 2009
+++ src/pcl/simple-streams/external-formats/cp1257.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1257.lisp,v 1.3 2009-06-21 15:12:25 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1257.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -30,5 +30,9 @@
311 299 316 353 324 326 243 333 245 246
247 371 322 347 363 252 380 382 729)))
-(define-external-format :cp1257 (:mac-roman)
+(define-external-format :cp1257 (:base :mac-roman :documentation
+"CP1257 is a Windows code page for Estonian, Latvian, and Lithuanian.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1257+)))
Index: src/pcl/simple-streams/external-formats/cp1258.lisp
diff -u src/pcl/simple-streams/external-formats/cp1258.lisp:1.3 src/pcl/simple-streams/external-formats/cp1258.lisp:1.4
--- src/pcl/simple-streams/external-formats/cp1258.lisp:1.3 Sun Jun 21 11:12:25 2009
+++ src/pcl/simple-streams/external-formats/cp1258.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1258.lisp,v 1.3 2009-06-21 15:12:25 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cp1258.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -30,5 +30,9 @@
237 238 239 273 241 803 243 244 417 246
247 248 249 250 251 252 432 8363 255)))
-(define-external-format :cp1258 (:mac-roman)
+(define-external-format :cp1258 (:base :mac-roman :documentation
+"CP1258 is a Windows code page for Vietnamese.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +ms-cp1258+)))
Index: src/pcl/simple-streams/external-formats/cr.lisp
diff -u src/pcl/simple-streams/external-formats/cr.lisp:1.2 src/pcl/simple-streams/external-formats/cr.lisp:1.3
--- src/pcl/simple-streams/external-formats/cr.lisp:1.2 Wed Sep 9 11:51:28 2009
+++ src/pcl/simple-streams/external-formats/cr.lisp Mon Jul 12 09:58:42 2010
@@ -4,12 +4,15 @@
;;; This code was written by Raymond Toy and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cr.lisp,v 1.2 2009-09-09 15:51:28 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/cr.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
;; Convert CR to/from #\newline.
-(define-composing-external-format :cr (:size 1)
+(define-composing-external-format :cr (:size 1 :documentation
+"CR is a composing external format that converts CR (carriage return)
+to a Lisp newline character. This is the Mac end-of-line character
+sequence.")
(input (state input unput tmp1 tmp2)
`(multiple-value-bind (,tmp1 ,tmp2)
,input
Index: src/pcl/simple-streams/external-formats/crlf.lisp
diff -u src/pcl/simple-streams/external-formats/crlf.lisp:1.3 src/pcl/simple-streams/external-formats/crlf.lisp:1.4
--- src/pcl/simple-streams/external-formats/crlf.lisp:1.3 Wed Sep 9 11:51:28 2009
+++ src/pcl/simple-streams/external-formats/crlf.lisp Mon Jul 12 09:58:42 2010
@@ -4,12 +4,15 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/crlf.lisp,v 1.3 2009-09-09 15:51:28 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/crlf.lisp,v 1.4 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
;; Convert DOS cr/lf end-of-line sequence to/from #\newline
-(define-composing-external-format :crlf (:max 2)
+(define-composing-external-format :crlf (:max 2 :documentation
+"CRLF is a composing external format that converts CR/LF (carriage
+return/linefeed) to a Lisp newline character. This is the Windoes
+end-of-line character sequence.")
(input (state input unput tmp1 tmp2 tmp3 tmp4)
`(multiple-value-bind (,tmp1 ,tmp2)
,input
Index: src/pcl/simple-streams/external-formats/final-sigma.lisp
diff -u src/pcl/simple-streams/external-formats/final-sigma.lisp:1.2 src/pcl/simple-streams/external-formats/final-sigma.lisp:1.3
--- src/pcl/simple-streams/external-formats/final-sigma.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/final-sigma.lisp Mon Jul 12 09:58:42 2010
@@ -4,12 +4,14 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/final-sigma.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/final-sigma.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
;; This is a composing format that attempts to detect sigma in
;; word-final position and change it from "σ" (U+03C3) to "ς" (U+03C2).
-(define-composing-external-format :final-sigma (:size 1)
+(define-composing-external-format :final-sigma (:size 1 :documentation
+"FINAL-SIGMA is a composing external format that attempts to detect
+sigma in word-final position and changes it from U+03C3 to U+03C2.")
(input (state input unput tmp1 tmp2 tmp3 tmp4)
`(multiple-value-bind (,tmp1 ,tmp2) ,input
(when (= ,tmp1 #x03C3)
Index: src/pcl/simple-streams/external-formats/iso8859-1.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-1.lisp:1.4 src/pcl/simple-streams/external-formats/iso8859-1.lisp:1.5
--- src/pcl/simple-streams/external-formats/iso8859-1.lisp:1.4 Wed Jun 30 00:02:53 2010
+++ src/pcl/simple-streams/external-formats/iso8859-1.lisp Mon Jul 12 09:58:42 2010
@@ -4,17 +4,27 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-1.lisp,v 1.4 2010-06-30 04:02:53 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-1.lisp,v 1.5 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
;; This is actually implemented in the external-formats code
;; It appears here only for reference, and will never get loaded
-(define-external-format :iso8859-1 (:size 1)
- ()
+(define-external-format :iso8859-1 (:size 1 :documentation
+"ISO8859-1 is an 8-bit character encoding generally intended for
+Western European languages including English, German, Italian,
+Norwegian, Portuguese, Spanish, Swedish and many others.
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
+ ()
(octets-to-code (state input unput error)
- (values ,input 1))
+ `(values ,input 1))
(code-to-octets (code state output error)
- (,output (if (> ,code 255) #x3F ,code))))
+ `(,output (if (> ,code 255)
+ (if ,error
+ (funcall ,error "Cannot output codepoint #x~X to ISO8859-1 stream"
+ ,code 1)
+ #x3F)
+ ,code))))
Index: src/pcl/simple-streams/external-formats/iso8859-10.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-10.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-10.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-10.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-10.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-10.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-10.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,10 @@
238 239 240 326 333 243 244 245 246 361 248 371 250
251 252 253 254 312)))
-(define-external-format :iso8859-10 (:iso8859-2)
+(define-external-format :iso8859-10 (:base :iso8859-2 :documentation
+"ISO8859-10 is an 8-bit character encoding intended to cover Nordic
+languages.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-10+)))
Index: src/pcl/simple-streams/external-formats/iso8859-13.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-13.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-13.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-13.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-13.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-13.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-13.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,9 @@
299 316 353 324 326 243 333 245 246 247 371 322 347
363 252 380 382 8217)))
-(define-external-format :iso8859-13 (:iso8859-2)
+(define-external-format :iso8859-13 (:base :iso8859-2 :documentation
+"ISO8859-13 is an 8-bit character encoding for the Baltic languages.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-13+)))
Index: src/pcl/simple-streams/external-formats/iso8859-14.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-14.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-14.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-14.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-14.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-14.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-14.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,10 @@
234 235 236 237 238 239 373 241 242 243 244 245 246
7787 248 249 250 251 252 253 375 255)))
-(define-external-format :iso8859-14 (:iso8859-2)
+(define-external-format :iso8859-14 (:base :iso8859-2 :documentation
+"ISO8859-14 is an 8-bit character encoding intended for the Celtic
+languages such as Irish, Manx, Scottish Gaelic, Welsh, and Breton.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-14+)))
Index: src/pcl/simple-streams/external-formats/iso8859-15.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-15.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-15.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-15.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-15.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-15.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-15.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,11 @@
238 239 240 241 242 243 244 245 246 247 248 249 250
251 252 253 254 255)))
-(define-external-format :iso8859-15 (:iso8859-2)
+(define-external-format :iso8859-15 (:base :iso8859-2 :documentation
+"ISO8859-15 is an 8-bit character encoding similar to ISO8859-1 but
+replaces some less common symbols with others, including adding the
+Euro sign.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-15+)))
Index: src/pcl/simple-streams/external-formats/iso8859-2.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-2.lisp:1.5 src/pcl/simple-streams/external-formats/iso8859-2.lisp:1.6
--- src/pcl/simple-streams/external-formats/iso8859-2.lisp:1.5 Wed Jul 7 17:59:49 2010
+++ src/pcl/simple-streams/external-formats/iso8859-2.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-2.lisp,v 1.5 2010-07-07 21:59:49 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-2.lisp,v 1.6 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,7 +19,14 @@
238 271 273 324 328 243 244 337 246 247 345 367 250
369 252 253 355 729)))
-(define-external-format :iso8859-2 (:size 1)
+(define-external-format :iso8859-2 (:size 1 :documentation
+"ISO8859-2 is an 8-bit character encoding generally intended for
+Eastern European languages including Bosnian, Croation, Czech, German,
+Hungarian, Polish, Romanian, Serbian Latin, Slovak, Slovene, Upper
+Sorbian, and Lower Sorbian.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-2+ :type (simple-array (unsigned-byte 16) (96)))
(itable (invert-table table) :type lisp::ntrie16))
Index: src/pcl/simple-streams/external-formats/iso8859-3.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-3.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-3.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-3.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-3.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-3.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-3.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,11 @@
237 238 239 65534 241 242 243 244 289 246 247 285 249
250 251 252 365 349 729)))
-(define-external-format :iso8859-3 (:iso8859-2)
+(define-external-format :iso8859-3 (:base :iso8859-2 :documentation
+"ISO8859-3 is an 8-bit character encoding intended for South European
+languages including Turkish, Maltese, and Esperanto. For Turkish,
+ISO8859-9 supersedes ISO8859-3.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-3+)))
Index: src/pcl/simple-streams/external-formats/iso8859-4.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-4.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-4.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-4.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-4.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-4.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-4.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,10 @@
238 299 273 326 333 311 244 245 246 247 248 371 250
251 252 361 363 729)))
-(define-external-format :iso8859-4 (:iso8859-2)
+(define-external-format :iso8859-4 (:base :iso8859-2 :documentation
+"ISO8859-4 is an 8-bit character encoding for North European languages
+including Estonian, Latvian, Lithuanian, Greenlandic, and Sami.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-4+)))
Index: src/pcl/simple-streams/external-formats/iso8859-5.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-5.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-5.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-5.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-5.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-5.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-5.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -20,5 +20,11 @@
1101 1102 1103 8470 1105 1106 1107 1108 1109 1110 1111
1112 1113 1114 1115 1116 167 1118 1119)))
-(define-external-format :iso8859-5 (:iso8859-2)
+(define-external-format :iso8859-5 (:base :iso8859-2 :documentation
+"ISO8859-5, informallly referred to as Latin/Cyrillic, is an 8-bit
+character encoding intended for languages using a Cyrillic alphabet
+such as Bulgarian, Belarusian, Russian, Serbian, and Macedonian.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-5+)))
Index: src/pcl/simple-streams/external-formats/iso8859-6.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-6.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-6.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-6.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-6.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-6.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-6.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -21,5 +21,10 @@
1616 1617 1618 65534 65534 65534 65534 65534 65534
65534 65534 65534 65534 65534 65534 65534)))
-(define-external-format :iso8859-6 (:iso8859-2)
+(define-external-format :iso8859-6 (:base :iso8859-2 :documentation
+"ISO8859-6 is an 8-bit character encoding generally intended languages
+using the Arabic alphabet.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-6+)))
Index: src/pcl/simple-streams/external-formats/iso8859-7.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-7.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-7.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-7.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-7.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-7.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-7.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,10 @@
956 957 958 959 960 961 962 963 964 965 966 967 968
969 970 971 972 973 974 65534)))
-(define-external-format :iso8859-7 (:iso8859-2)
+(define-external-format :iso8859-7 (:base :iso8859-2 :documentation
+"ISO8859-7 is an 8-bit character encoding for the modern Greek
+language.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-7+)))
Index: src/pcl/simple-streams/external-formats/iso8859-8.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-8.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-8.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-8.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-8.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-8.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-8.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -21,5 +21,9 @@
1509 1510 1511 1512 1513 1514 65534 65534 8206 8207
65534)))
-(define-external-format :iso8859-8 (:iso8859-2)
+(define-external-format :iso8859-8 (:base :iso8859-2 :documentation
+"ISO8859-8 is an 8-bit character encoding intended for Hebrew.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-8+)))
Index: src/pcl/simple-streams/external-formats/iso8859-9.lisp
diff -u src/pcl/simple-streams/external-formats/iso8859-9.lisp:1.2 src/pcl/simple-streams/external-formats/iso8859-9.lisp:1.3
--- src/pcl/simple-streams/external-formats/iso8859-9.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/iso8859-9.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-9.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/iso8859-9.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -19,5 +19,9 @@
238 239 287 241 242 243 244 245 246 247 248 249 250
251 252 305 351 255)))
-(define-external-format :iso8859-9 (:iso8859-2)
+(define-external-format :iso8859-9 (:base :iso8859-2 :documentation
+"ISO8859-9 is an 8-bit character encoding for the Turkish language.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +iso-8859-9+)))
Index: src/pcl/simple-streams/external-formats/koi8-r.lisp
diff -u src/pcl/simple-streams/external-formats/koi8-r.lisp:1.2 src/pcl/simple-streams/external-formats/koi8-r.lisp:1.3
--- src/pcl/simple-streams/external-formats/koi8-r.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/koi8-r.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/koi8-r.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/koi8-r.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -24,5 +24,9 @@
1053 1054 1055 1071 1056 1057 1058 1059 1046 1042
1068 1067 1047 1064 1069 1065 1063 1066)))
-(define-external-format :koi8-r (:mac-roman)
+(define-external-format :koi8-r (:base :mac-roman :documentation
+"KOI8-R is an 8-bit character encoding designed to cover Russian.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +koi8-r+)))
Index: src/pcl/simple-streams/external-formats/mac-cyrillic.lisp
diff -u src/pcl/simple-streams/external-formats/mac-cyrillic.lisp:1.2 src/pcl/simple-streams/external-formats/mac-cyrillic.lisp:1.3
--- src/pcl/simple-streams/external-formats/mac-cyrillic.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/mac-cyrillic.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-cyrillic.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-cyrillic.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -23,5 +23,10 @@
1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097
1098 1099 1100 1101 1102 164)))
-(define-external-format :mac-cyrillic (:mac-roman)
+(define-external-format :mac-cyrillic (:base :mac-roman :documentation
+"MAC-CYRILLIC is an 8-bit character encoding for Cyrillic text on
+Apple Macintosh computers.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +mac-cyrillic+)))
Index: src/pcl/simple-streams/external-formats/mac-greek.lisp
diff -u src/pcl/simple-streams/external-formats/mac-greek.lisp:1.2 src/pcl/simple-streams/external-formats/mac-greek.lisp:1.3
--- src/pcl/simple-streams/external-formats/mac-greek.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/mac-greek.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-greek.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-greek.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -21,5 +21,10 @@
947 951 953 958 954 955 956 957 959 960 974 961 963
964 952 969 962 967 965 950 970 971 912 944 65534)))
-(define-external-format :mac-greek (:mac-roman)
+(define-external-format :mac-greek (:base :mac-roman :documentation
+"MAC-GREEK is an 8-bit character encoding for Greek text on Apple
+Macintosh computers.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +mac-greek+)))
Index: src/pcl/simple-streams/external-formats/mac-icelandic.lisp
diff -u src/pcl/simple-streams/external-formats/mac-icelandic.lisp:1.2 src/pcl/simple-streams/external-formats/mac-icelandic.lisp:1.3
--- src/pcl/simple-streams/external-formats/mac-icelandic.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/mac-icelandic.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-icelandic.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-icelandic.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -22,5 +22,10 @@
65534 210 218 219 217 305 710 732 175 728 729 730 184
733 731 711)))
-(define-external-format :mac-icelandic (:mac-roman)
+(define-external-format :mac-icelandic (:base :mac-roman :documentation
+"MAC-ICELANDIC is an 8-bit character encoding for Icelandic text on
+Apple Macintosh computers.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +mac-icelandic+)))
Index: src/pcl/simple-streams/external-formats/mac-latin2.lisp
diff -u src/pcl/simple-streams/external-formats/mac-latin2.lisp:1.2 src/pcl/simple-streams/external-formats/mac-latin2.lisp:1.3
--- src/pcl/simple-streams/external-formats/mac-latin2.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/mac-latin2.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-latin2.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-latin2.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -22,5 +22,11 @@
218 367 368 369 370 371 221 253 311 379 321 380 290
711)))
-(define-external-format :mac-latin2 (:mac-roman)
+(define-external-format :mac-latin2 (:base :mac-roman :documentation
+
+"MAC-LATIN2 is an 8-bit character encoding for Central European text
+on Apple Macintosh computers.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +mac-latin2+)))
Index: src/pcl/simple-streams/external-formats/mac-roman.lisp
diff -u src/pcl/simple-streams/external-formats/mac-roman.lisp:1.6 src/pcl/simple-streams/external-formats/mac-roman.lisp:1.7
--- src/pcl/simple-streams/external-formats/mac-roman.lisp:1.6 Wed Jul 7 17:59:49 2010
+++ src/pcl/simple-streams/external-formats/mac-roman.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-roman.lisp,v 1.6 2010-07-07 21:59:49 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-roman.lisp,v 1.7 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -24,7 +24,12 @@
212 63743 210 218 219 217 305 710 732 175 728 729 730
184 733 731 711)))
-(define-external-format :mac-roman (:size 1)
+(define-external-format :mac-roman (:size 1 :documentation
+"MAC-ROMAN is an 8-bit character encoding for Western European
+languages including English.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +mac-roman+ :type (simple-array (unsigned-byte 16) (128)))
(itable (invert-table table) :type lisp::ntrie16))
Index: src/pcl/simple-streams/external-formats/mac-turkish.lisp
diff -u src/pcl/simple-streams/external-formats/mac-turkish.lisp:1.2 src/pcl/simple-streams/external-formats/mac-turkish.lisp:1.3
--- src/pcl/simple-streams/external-formats/mac-turkish.lisp:1.2 Thu Jun 11 12:04:02 2009
+++ src/pcl/simple-streams/external-formats/mac-turkish.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-turkish.lisp,v 1.2 2009-06-11 16:04:02 rtoy Rel $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/mac-turkish.lisp,v 1.3 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -22,5 +22,11 @@
65534 210 218 219 217 65534 710 732 175 728 729 730
184 733 731 711)))
-(define-external-format :mac-turkish (:mac-roman)
+(define-external-format :mac-turkish (:base :mac-roman :documentation
+
+"MAC-TURKISH is an 8-bit character encoding for Turkish text on
+Apple Macintosh computers.
+
+By default, illegal inputs are replaced by the Unicode replacement
+character and illegal outputs are replaced by a question mark.")
((table +mac-turkish+)))
Index: src/pcl/simple-streams/external-formats/utf-16-be.lisp
diff -u src/pcl/simple-streams/external-formats/utf-16-be.lisp:1.8 src/pcl/simple-streams/external-formats/utf-16-be.lisp:1.9
--- src/pcl/simple-streams/external-formats/utf-16-be.lisp:1.8 Mon Jul 5 00:12:47 2010
+++ src/pcl/simple-streams/external-formats/utf-16-be.lisp Mon Jul 12 09:58:42 2010
@@ -4,13 +4,20 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-16-be.lisp,v 1.8 2010-07-05 04:12:47 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-16-be.lisp,v 1.9 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
;; UTF-16BE. BOM is not recognized and is never output.
-(define-external-format :utf-16-be (:size 2)
+(define-external-format :utf-16-be (:size 2 :documentation
+"UTF-16-BE is a variable length character encoding for Unicode. For
+both input and output, the data is assumed to be in big-endian order.
+No byte-order mark is allowed on input, and no byte-order mark is
+produced on output. (This is specified by the Unicode standard.)
+
+By default, illegal inputs and illegal outputs are replaced by the
+Unicode replacement character.")
()
(octets-to-code (state input unput error c1 c2 code next)
Index: src/pcl/simple-streams/external-formats/utf-16-le.lisp
diff -u src/pcl/simple-streams/external-formats/utf-16-le.lisp:1.8 src/pcl/simple-streams/external-formats/utf-16-le.lisp:1.9
--- src/pcl/simple-streams/external-formats/utf-16-le.lisp:1.8 Mon Jul 5 00:12:47 2010
+++ src/pcl/simple-streams/external-formats/utf-16-le.lisp Mon Jul 12 09:58:42 2010
@@ -4,14 +4,21 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-16-le.lisp,v 1.8 2010-07-05 04:12:47 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-16-le.lisp,v 1.9 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
;; UTF-16LE. BOM is not recognized, and is never output.
;;
;; The state is either NIL or a codepoint.
-(define-external-format :utf-16-le (:size 2)
+(define-external-format :utf-16-le (:size 2 :documentation
+"UTF-16-LE is a variable length character encoding for Unicode. For
+both input and output, the data is assumed to be in little-endian
+order. No byte-order mark is allowed on input, and no byte-order mark
+is produced on output. (This is specified by the Unicode standard.)
+
+By default, illegal inputs and illegal outputs are replaced by the
+Unicode replacement character.")
()
(octets-to-code (state input unput error c1 c2 code next)
Index: src/pcl/simple-streams/external-formats/utf-16.lisp
diff -u src/pcl/simple-streams/external-formats/utf-16.lisp:1.10 src/pcl/simple-streams/external-formats/utf-16.lisp:1.11
--- src/pcl/simple-streams/external-formats/utf-16.lisp:1.10 Mon Jul 5 00:12:47 2010
+++ src/pcl/simple-streams/external-formats/utf-16.lisp Mon Jul 12 09:58:42 2010
@@ -1,7 +1,7 @@
;;; -*- Mode: LISP; Syntax: ANSI-Common-Lisp; Package: STREAM -*-
;;;
;;; **********************************************************************
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-16.lisp,v 1.10 2010-07-05 04:12:47 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-16.lisp,v 1.11 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -22,7 +22,15 @@
;;
;; When writing, never output a BOM.
-(define-external-format :utf-16 (:size 2)
+(define-external-format :utf-16 (:size 2 :documentation
+"UTF-16 is a variable length character encoding for Unicode. On
+input, a byte-order mark is recognized. If no byte-order mark is
+given on input, then the encoding is assumed to be big-endian. For
+output, the byte-order mark is not written, and the output is
+big-endian. (This is specified by the Unicode standard.)
+
+By default, illegal inputs and illegal outputs are replaced by the
+Unicode replacement character.")
()
(octets-to-code (state input unput error c1 c2 code wd next st)
Index: src/pcl/simple-streams/external-formats/utf-32-be.lisp
diff -u src/pcl/simple-streams/external-formats/utf-32-be.lisp:1.7 src/pcl/simple-streams/external-formats/utf-32-be.lisp:1.8
--- src/pcl/simple-streams/external-formats/utf-32-be.lisp:1.7 Mon Jul 5 00:12:47 2010
+++ src/pcl/simple-streams/external-formats/utf-32-be.lisp Mon Jul 12 09:58:42 2010
@@ -4,11 +4,19 @@
;;; This code was written by Raymond Toy and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-32-be.lisp,v 1.7 2010-07-05 04:12:47 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-32-be.lisp,v 1.8 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
-(define-external-format :utf-32-be (:size 4)
+(define-external-format :utf-32-be (:size 4 :documentation
+"UTF-32-BE is a fixed-length character encoding of 4 octets for
+Unicode. For both input and output, the data is assumed to be in
+big-endian order. No byte-order mark is allowed on input, and no
+byte-order mark is produced on output. (This is specified by the
+Unicode standard.)
+
+By default, illegal inputs and illegal outputs are replaced by the
+Unicode replacement character.")
()
(octets-to-code (state input unput error c c1 c2 c3 c4)
Index: src/pcl/simple-streams/external-formats/utf-32-le.lisp
diff -u src/pcl/simple-streams/external-formats/utf-32-le.lisp:1.7 src/pcl/simple-streams/external-formats/utf-32-le.lisp:1.8
--- src/pcl/simple-streams/external-formats/utf-32-le.lisp:1.7 Mon Jul 5 00:12:47 2010
+++ src/pcl/simple-streams/external-formats/utf-32-le.lisp Mon Jul 12 09:58:42 2010
@@ -4,11 +4,19 @@
;;; This code was written by Raymond Toy and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-32-le.lisp,v 1.7 2010-07-05 04:12:47 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-32-le.lisp,v 1.8 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
-(define-external-format :utf-32-le (:size 4)
+(define-external-format :utf-32-le (:size 4 :documentation
+"UTF-32-LE is a fixed-length character encoding of 4 octets for
+Unicode. For both input and output, the data is assumed to be in
+little-endian order. No byte-order mark is allowed on input, and no
+byte-order mark is produced on output. (This is specified by the
+Unicode standard.)
+
+By default, illegal inputs and illegal outputs are replaced by the
+Unicode replacement character.")
()
(octets-to-code (state input unput error c c1 c2 c3 c4)
Index: src/pcl/simple-streams/external-formats/utf-32.lisp
diff -u src/pcl/simple-streams/external-formats/utf-32.lisp:1.9 src/pcl/simple-streams/external-formats/utf-32.lisp:1.10
--- src/pcl/simple-streams/external-formats/utf-32.lisp:1.9 Mon Jul 5 00:12:47 2010
+++ src/pcl/simple-streams/external-formats/utf-32.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Raymond Toy and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-32.lisp,v 1.9 2010-07-05 04:12:47 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-32.lisp,v 1.10 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -26,7 +26,15 @@
;; (evenp state) = big-endian
;; (zerop state) = #xFEFF/#xFFFE is BOM (to be skipped)
;;
-(define-external-format :utf-32 (:size 4)
+(define-external-format :utf-32 (:size 4 :documentation
+"UTF-32 is a fixed-length character encoding of 4 octets for Unicode.
+On input, a byte-order mark is recognized. If no byte-order mark is
+given on input, then the encoding is assumed to be big-endian. For
+output, the byte-order mark is not written, and the output is
+big-endian. (This is specified by the Unicode standard.)
+
+By default, illegal inputs and illegal outputs are replaced by the
+Unicode replacement character.")
()
(octets-to-code (state input unput code error c1 c2 c3 c4 st wd)
Index: src/pcl/simple-streams/external-formats/utf-8.lisp
diff -u src/pcl/simple-streams/external-formats/utf-8.lisp:1.12 src/pcl/simple-streams/external-formats/utf-8.lisp:1.13
--- src/pcl/simple-streams/external-formats/utf-8.lisp:1.12 Wed Jul 7 09:43:12 2010
+++ src/pcl/simple-streams/external-formats/utf-8.lisp Mon Jul 12 09:58:42 2010
@@ -4,7 +4,7 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-8.lisp,v 1.12 2010-07-07 13:43:12 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/utf-8.lisp,v 1.13 2010-07-12 13:58:42 rtoy Exp $")
(in-package "STREAM")
@@ -13,7 +13,11 @@
;;
;; See Table 3-7, Ch 3.9 in the Unicode book.
-(define-external-format :utf-8 (:min 1 :max 4)
+(define-external-format :utf-8 (:min 1 :max 4 :documentation
+"UTF-8 is a variable-length character encoding for Unicode. By
+default, illegal input sequences are replaced by the Unicode
+replacement character.")
+
()
(octets-to-code (state input unput error c i j n)
`(labels ((utf8 (,c ,i)
Index: src/pcl/simple-streams/external-formats/void.lisp
diff -u src/pcl/simple-streams/external-formats/void.lisp:1.4 src/pcl/simple-streams/external-formats/void.lisp:1.5
--- src/pcl/simple-streams/external-formats/void.lisp:1.4 Wed Jun 30 00:02:53 2010
+++ src/pcl/simple-streams/external-formats/void.lisp Mon Jul 12 09:58:42 2010
@@ -4,12 +4,13 @@
;;; This code was written by Paul Foley and has been placed in the public
;;; domain.
;;;
-(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/void.lisp,v 1.4 2010-06-30 04:02:53 rtoy Exp $")
+(ext:file-comment "$Header: /project/cmucl/cvsroot/src/pcl/simple-streams/external-formats/void.lisp,v 1.5 2010-07-12 13:58:42 rtoy Exp $")
;; This is actually implemented in the external-formats code
;; It appears here only for reference, and will never get loaded
-(define-external-format :void (:size 0)
+(define-external-format :void (:size 0 :documentation
+"Void external format that signals an error on any input or output.")
()
(octets-to-code (state input unput error)
`(error 'void-external-format))
More information about the cmucl-commit
mailing list