Network Working GroupInternet Engineering Task Force (IETF) Y.YONEYA Internet-DraftYoneya Request for Comments: 7790 JPRSIntended status:Category: Informational T. NemotoExpires: May 4, 2016ISSN: 2070-1721 Keio UniversityNovember 1, 2015February 2016 MappingcharactersCharacters forPRECIS classes draft-ietf-precis-mappings-12Classes of the Preparation, Enforcement, and Comparison of Internationalized Strings (PRECIS) Abstract The framework for the preparation, enforcement, and comparison of internationalized strings("PRECIS")(PRECIS) defines several classes of strings for use in application protocols. Because many protocols perform case-sensitive or case-insensitive string comparison, it is necessary to define methods for case mapping. In addition, both the Internationalized Domain Names in Applications (IDNA) and the PRECIS problem statement describe mappings for internationalized strings that are not limited to case, but include width mapping and mapping of delimiters and other special characters that can be taken into consideration. This document provides guidelines for designers of PRECIS profiles and describes several mappings that can be applied between receiving user input and passing permitted code points to internationalized protocols. In particular, this document describes both locale-dependent and context-depending case mappings as well as additional mappings for delimiters and special characters. Status of This Memo ThisInternet-Draftdocument issubmitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documentsnot an Internet Standards Track specification; it is published for informational purposes. This document is a product of the Internet Engineering Task Force (IETF).Note that other groups may also distribute working documents as Internet-Drafts. The listIt represents the consensus ofcurrent Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents validthe IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are amaximumcandidate for any level of Internet Standard; see Section 2 of RFC 5741. Information about the current status ofsix monthsthis document, any errata, and how to provide feedback on it may beupdated, replaced, or obsoleted by other documentsobtained atany time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on May 4, 2016.http://www.rfc-editor.org/info/rfc7790. Copyright Notice Copyright (c)20152016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . .32 2.Protocol dependent mappingsProtocol-Dependent Mappings . . . . . . . . . . . . . . . . . 3 2.1. DelimitermappingMapping . . . . . . . . . . . . . . . . . . . . 3 2.2. SpecialmappingMapping . . . . . . . . . . . . . . . . . . . . .43 2.3. Localcase mappingCase Mapping . . . . . . . . . . . . . . . . . . . 4 3. Order ofoperationsOperations . . . . . . . . . . . . . . . . . . . . . 5 4. Security Considerations . . . . . . . . . . . . . . . . . . . 5 5.IANA Considerations . . . . . . . . . . . . . . . . . . .References . .6 6. Acknowledgment. . . . . . . . . . . . . . . . . . . . . . .6 7.5 5.1. Normative References . . . . . . . . . . . . . . . . . .. . . . . . . 6 7.1. Normative5 5.2. Informative References . . . . . . . . . . . . . . . . ..67.2. Informative References . . . . . . . . . .Appendix A. Mapping Type List . . . . . . .6 Appendix A. Mapping type list each protocol. . . . . . . . . .87 A.1. Mappingtype listType List foreach protocolEach Protocol . . . . . . . . . . .87 Appendix B.The reason why local case mapping is alternativeWhy Local Case Mapping Is an Alternative tocase mappingCase Mapping in the PRECISframeworkFramework . . . . . . . . . . 8 Appendix C.Limitation to local case mapping . . . . . . . . . . 9 Appendix D. Change Log . . . .Limitations of Local Case Mapping . . . . . . . . . 8 Acknowledgments . . . . . . . .9 D.1. Changes since -00. . . . . . . . . . . . . . . . . 9 Authors' Addresses . . .9 D.2. Changes since -01. . . . . . . . . . . . . . . . . . . . 9D.3. Changes since -02 . . . . . . . . . . . . . . . . . . . . 10 D.4. Changes since -03 . . . . . . . . . . . . . . . . . . . . 10 D.5. Changes since -04 . . . . . . . . . . . . . . . . . . . . 10 D.6. Changes since -05 . . . . . . . . . . . . . . . . . . . . 10 D.7. Changes since -06 . . . . . . . . . . . . . . . . . . . . 11 D.8. Changes since -07 . . . . . . . . . . . . . . . . . . . . 11 D.9. Changes since -08 . . . . . . . . . . . . . . . . . . . . 11 D.10. Changes since -09 . . . . . . . . . . . . . . . . . . . . 11 D.11. Changes since -10 . . . . . . . . . . . . . . . . . . . . 12 D.12. Changes since -11 . . . . . . . . . . . . . . . . . . . . 12 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 131. Introduction In many cases, user input of internationalized strings is generated through the use of an input method editor ("IME") or through copy- and-paste from free text. Users generally do not care about the case and/or width of input characters because they consider those characters to be functionally equivalent or visually identical. Furthermore, users rarely switch the IME state to input special characters such as protocol elements. For Internationalized Domain Names("IDNs"),(IDNs), the IDNA Mapping specification [RFC5895] describes methods for handling these issues. For PRECIS strings, case mapping and width mapping are defined in the PRECIS framework specification [RFC7564]. The case and width mappings defined in the PRECIS framework do not handle other mappings such as delimiter characters, special characters, and locale-dependent or context-dependentcase;cases; these mappings are also important in order to increase the probability that the resulting strings compare as users expect. This document provides guidelines for authors of protocol profiles of the PRECIS framework and describes several mappings that can be applied between receiving user input and passing permitted code points to internationalized protocols. The delimiter mapping and special mapping rules described here are applied as "additional mappings" beyond those defined in the PRECIS framework, whereas the "local case mapping" rule provides locale-dependent and context-dependent alternative case mappings for specific target characters. 2.Protocol dependent mappingsProtocol-Dependent Mappings The PRECIS framework defines several protocol-independent mappings. The additional mappings and local case mapping defined in this document areprotocol-dependent,protocol dependent, i.e., they depend on the rules for a particular application protocol. 2.1. DelimitermappingMapping Some application protocols define delimiters for their own use, resulting in the fact that the delimiters are different for each protocol. The delimiter mapping table should therefore be based on a well-defined mapping table for each protocol. Delimiter mapping is used to map characters that are similar to protocol delimiters into the canonical delimiter characters. For example, there are width-compatible characters that correspond to the '@' in email addresses and the ':' and '/' in URIs. The '+', '-', '<' and '>' characters are other common delimiters that might require such mapping. For the FULL STOP character (U+002E), a delimiter in the visual presentation of domain names, some IMEs produce a character such as IDEOGRAPHIC FULL STOP (U+3002) when a user types FULL STOP on the keyboard. In all these cases, the visually similar characters that can come from user input need to be mapped to the correct protocol delimiter characters before the string is passed to the protocol. 2.2. SpecialmappingMapping Aside from delimiter characters, certain protocols have characters which need to be mapped in ways that are different from the rules specified in the PRECIS framework (e.g., mapping non-ASCII space characters to ASCII space). In this document, these mappings are called "special mappings". They are different for each protocol. Therefore, the special mapping table should be based on a well- defined mapping table for each protocol. Examples of special mapping are the following; o White spaces such as CHARACTERTABULATION(U+0009)TABULATION (U+0009) or IDEOGRAPHICSPACE(U+3000)SPACE (U+3000) are mapped to SPACE (U+0020) o Some characters such as control characters are mapped to nothing (Deletion) As examples,EAPthe Extensible Authentication Protocol (EAP) [RFC3748], SASLprep [RFC4013], IMAP4ACL [RFC4314]Access Control List (ACL) [RFC4314], and LDAPprep [RFC4518] define the rule that some codepoints for thenon-ASCIInon- ASCII space are mapped to SPACE (U+0020). 2.3. Localcase mapping TheCase Mapping The purpose of local case mapping is to increase the probability of results that users expect when character case is changed (e.g., map uppercase to lowercase) between input and use in a protocol. Local case mapping selectively affects characters whose case mapping depends on locale and/or context. (Note: The term "locale" in this document practically means "language" or "language and region" because the locale based on that language configuration of applications on POSIX is selected by "locale"information and referredinformation. See also the "Note" insectionSection 2.1.1 ofBCP 47RFC 5646 [RFC5646].) As an example oflocalelocale- and context-dependent mapping, LATIN CAPITAL LETTER I ("I", U+0049) is normally mapped to LATIN SMALL LETTER I ("i", U+0069); however, if the language is Turkish (or one of several other languages), unless an I is before a dot_above, the character should be mapped to LATIN SMALL LETTER DOTLESS I (U+0131). Case mapping using Unicode Default Case Folding in the PRECIS framework does not consider such locale or context because it is a common framework for internationalization. Local case mapping defined in this documentcorrespondscorrespond to demands from applicationswhich supportsthat support users' locale and/or context. The complete set of possible target characters for local case mapping are the characters specified intheSpecialCasing.txt [Specialcasing]fileintheSection 3.13 of the Unicode Standard [Unicode], but the specific set of target characters selected for local case mapping depends on locale and/or context, as further explained inthe SpecialCasing.txt file.SpecialCasing.txt. Thecase foldingcase-folding method for a selected target character is to map intolower caselowercase as defined in SpecialCasing.txt. Thecase foldingcase-folding method for all other, non-target characters is as specified intheSection 5.2.3 of the PRECISframework .framework. When an application supports users' locale and/or context, use of local case mapping can increase the probability that string comparisons yield the results that users expect. If a PRECIS profile selects Unicode Default Case Folding as the preferred method of case mapping, the profile designers may consider whether local case mapping can be applied.AndAnd, if it can be applied, it is better to add "alternatively, local case mapping might be applicable" after "Unicode Default Case Folding" so that application developers are aware of the alternative. SeetheAppendix B for a description of why local case mapping can be an alternative. 3. Order ofoperationsOperations Delimiter mapping and special mapping as described in this document are expected to be applied as the "Additional Mapping Rule" mentioned intheSection 5.2.2 of the PRECIS framework. Although thedelimiteddelimiter mapping and special mapping could be applied in either order, this document recommends the following order to minimize the effect ofcode pointcode-point changes introduced by the mappings and to be acceptable to the widest user community: 1. Delimiter mapping 2. Special mapping 4. Security Considerations Detailed security considerations for PRECIS strings are discussed in the PRECIS framework specification [RFC7564]. This document inherits the considerations as well. As with Mapping Characters for IDNA2008 [RFC5895], this document suggests creating mappings that might cause confusion for some users while alleviating confusioninfor other users. Such confusion is not covered in any depth in this document. 5.IANA Considerations This document has no actions for the IANA. 6. Acknowledgment Martin Duerst suggested a need for the case folding about the mapping (map final sigma to sigma, German sz to ss,.). Alexey Melnikov, Andrew Sullivan, Barry Leiba, David Black, Heather Flanagan, Joe Hildebrand, John Klensin, Marc Blanchet, Pete Resnick and Peter Saint-Andre, et al. gave important suggestion for this document during working group discussions. 7.References7.1.5.1. Normative References [RFC7564] Saint-Andre, P. and M. Blanchet, "PRECIS Framework: Preparation, Enforcement, and Comparison of Internationalized Strings in Application Protocols", RFC 7564, DOI 10.17487/RFC7564, May 2015, <http://www.rfc-editor.org/info/rfc7564>. [Unicode] The Unicode Consortium, "The Unicode Standard, Version 7.0.0",<http://www.unicode.org/versions/Unicode7.0.0/>, 2012.(Mountain View, CA: The Unicode Consortium, 2014. ISBN 978-1-936213-09-2), <http://www.unicode.org/versions/Unicode7.0.0/>. [Casefolding] The Unicode Consortium, "CaseFolding-7.0.0.txt", Unicode Character Database, July 2011, <http://www.unicode.org/Public/7.0.0/ucd/CaseFolding.txt>. [Specialcasing] The Unicode Consortium, "SpecialCasing-7.0.0.txt", Unicode Character Database, July 2011, <http://www.unicode.org/Public/7.0.0/ucd/ SpecialCasing.txt>.7.2.5.2. Informative References [RFC3454] Hoffman, P. and M. Blanchet, "Preparation of Internationalized Strings ("stringprep")", RFC 3454, DOI 10.17487/RFC3454, December 2002, <http://www.rfc-editor.org/info/rfc3454>. [RFC3490] Faltstrom, P., Hoffman, P., and A. Costello, "Internationalizing Domain Names in Applications (IDNA)", RFC 3490, DOI 10.17487/RFC3490, March 2003, <http://www.rfc-editor.org/info/rfc3490>. [RFC3491] Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep Profile for Internationalized Domain Names (IDN)", RFC 3491, DOI 10.17487/RFC3491, March 2003, <http://www.rfc-editor.org/info/rfc3491>. [RFC3722] Bakke, M., "String Profile for Internet Small Computer Systems Interface (iSCSI) Names", RFC 3722, DOI 10.17487/RFC3722, April 2004, <http://www.rfc-editor.org/info/rfc3722>. [RFC3748] Aboba, B., Blunk, L., Vollbrecht, J., Carlson, J., and H. Levkowetz, Ed., "Extensible Authentication Protocol (EAP)", RFC 3748, DOI 10.17487/RFC3748, June 2004, <http://www.rfc-editor.org/info/rfc3748>. [RFC4013] Zeilenga, K., "SASLprep: Stringprep Profile for User Names and Passwords", RFC 4013, DOI 10.17487/RFC4013, February 2005, <http://www.rfc-editor.org/info/rfc4013>. [RFC4314] Melnikov, A., "IMAP4 Access Control List (ACL) Extension", RFC 4314, DOI 10.17487/RFC4314, December 2005, <http://www.rfc-editor.org/info/rfc4314>. [RFC4518] Zeilenga, K., "Lightweight Directory Access Protocol (LDAP): Internationalized String Preparation", RFC 4518, DOI 10.17487/RFC4518, June 2006, <http://www.rfc-editor.org/info/rfc4518>. [RFC5646] Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying Languages", BCP 47, RFC 5646, DOI 10.17487/RFC5646, September 2009, <http://www.rfc-editor.org/info/rfc5646>. [RFC5895] Resnick, P. and P. Hoffman, "Mapping Characters for Internationalized Domain Names in Applications (IDNA) 2008", RFC 5895, DOI 10.17487/RFC5895, September 2010, <http://www.rfc-editor.org/info/rfc5895>. [RFC6122] Saint-Andre, P., "Extensible Messaging and Presence Protocol (XMPP): Address Format", RFC 6122, DOI 10.17487/RFC6122, March 2011, <http://www.rfc-editor.org/info/rfc6122>. [RFC6885] Blanchet, M. and A. Sullivan, "Stringprep Revision and Problem Statement for the Preparation and Comparison of Internationalized Strings (PRECIS)", RFC 6885, DOI 10.17487/RFC6885, March 2013, <http://www.rfc-editor.org/info/rfc6885>. [ISO.3166-1] International Organization for Standardization, "Codes for the representation of names of countries and their subdivisions - Part 1: Country codes", ISO Standard3166- 1:1997,3166-1:1997, 1997. Appendix A. Mappingtype list each protocolType List A.1. Mappingtype listType List foreach protocolEach Protocol This table is the mapping type list for each protocol. Values marked "o" indicate that the protocoluseuses the type of mapping. Values marked "-" indicate that the protocol doesn't use the type of mapping.+----------------------+-------------+-----------+------+---------++---------------------+-------------+-----------+------+---------+ | Protocol and | Width | Delimiter | Case | Special | | mapping RFC | (NFKC) | | | |+----------------------+-------------+-----------+------+---------++---------------------+-------------+-----------+------+---------+ | IDNA(RFC 3490)[RFC3490] | - | o | - | - | | IDNA(RFC 3491)[RFC3491] | o | - | o | - | | iSCSI(RFC 3722)[RFC3722] | o | - | o | - | | EAP(RFC 3748)[RFC3748] | o | - | - | o | | IMAP(RFC 4314)[RFC4314] | o | - | - | o | | LDAP(RFC 4518)[RFC4518] | o | - | o | o |+----------------------+-------------+-----------+------+---------++---------------------+-------------+-----------+------+---------+ Appendix B.The reason why local case mapping is alternativeWhy Local Case Mapping Is an Alternative tocase mappingCase Mapping in the PRECISframeworkFramework Local case mapping is an alternative to Unicode Default Case Folding instead of being applied sequentially.Because, oneOne outstanding issue regarding full case folding for charactersis,is that some lowercase characters like "LATIN SMALL LETTER SHARP S" (U+00DF) (hereinafter referred to as "eszett") and ligatures like "LATIN SMALL LIGATURE FF" (U+FB00) that are described in the "Unconditional mappings" sectionUnconditional mappingsof SpecialCasing.txt become a different codepointby performingwhen the case mapping is performed using Unicode Default Case Folding in the PRECIS framework. In particular, German's eszett can not keep the locale because eszett becomes two "LATIN SMALL LETTER S"s (U+0073 U+0073)by performingwhen the case mapping is performed using Unicode Default Case Folding. On the other hand, eszett doesn't become a different codepointbywhen performing the case mapping in SpecialCasing.txt. Therefore, if it is necessary to keep the locale of characters, PRECIS profile designers should select local case mapping as an alternative to Unicode Default Case Folding. Appendix C.Limitation to local case mappingLimitations of Local Case Mapping As described inthe sectionSection 2.3, the possible target characters of local case mapping are specified in SpecialCasing.txt. The Unicode Standard (at least, up to version 7.0.0) does not define anycontext-dependentcontext- dependent mappings between "GREEK SMALL LETTER SIGMA" (U+03C3) (hereinafter referred to as "small sigma") and "GREEK SMALL LETTER FINAL SIGMA" (U+03C2) (hereinafter referred to as "final sigma"). Thus, local case mapping is not applicable to small sigma or final sigma, so case mapping in the PRECIS framework always maps final sigma to small sigma, independent of context, as also specified by Unicode Default Case Folding.(Note: FollowingThe following comments are fromSpecialCasing.txt.)SpecialCasing.txt. (Line breaks have been added due to line-length limitations.) # Note: the following cases are not included, since they would case-fold in lowercasing # 03C3; 03C2; 03A3; 03A3; Final_Sigma; # GREEK SMALL LETTER SIGMA # 03C2; 03C3; 03A3; 03A3; Not_Final_Sigma; # GREEK SMALL LETTERAppendix D. Change Log D.1. Changes since -00 o Modify the Section 4.3 "Local case mapping" to specifyFINAL SIGMA Acknowledgments Martin Duerst suggested a need for themethod to calculate codepoints that localcasemapping targets. o Add the Section 6 "Open issues". o Modify the Section 7 "IANA Considerations". o Modify the Section 8 "Security Considerations". o Removefolding about the"The initial PRECIS local casemappingregistrations". o Add the Appendix C "Code points list(map final sigma to sigma, German sz to ss, etc.). Alexey Melnikov, Andrew Sullivan, Barry Leiba, David Black, Heather Flanagan, Joe Hildebrand, John Klensin, Marc Blanchet, Pete Resnick, and Peter Saint-Andre, et al., gave important suggestions forlocal case mapping". o Add the Appendix D "Change Log". D.2. Changes since -01 o Unified PRECIS notation in all capital letters as well as other documents. o Removed the Section 1 "Types of mapping" and the Section 2 "Protocol independent mapping" because width mapping is now in framework document. o Added relationship between the framework document andthis documentin the Section 3 "Order of operations". o Updated the Section 4 "Open issues" to address new issue raised on mailing list. o Move the Section 6 "IANA Considerations" after the Section 5 "Security Considerations". o Remove the Appendix B "Codepoints which need special mapping" and mentioned related documents in the Section 2.2 . D.3. Changes since -02 o Removed the "Open issues". D.4. Changes since -03 o Modify the Section 1 "Introduction" in more clear text. o Modify the Section 2.3 "Local case mapping" to clarify the purpose of the local case mapping and an example, and add restriction to use with PRECIS framework. o Change the format in the Appendix B "Code points list for local case mapping". o Split the Section 7 "References" into "Normative References" and "Informative References" o Update the Unicode version 6.2 to 6.3 in this document. D.5. Changes since -04 o Correct a sentence in the Section 2.3 "Local case mapping". D.6. Changes since -05 o Correct some sentences in this document. o Modify the local case mapping's rule and target characters in the Section 2.3 "Local case mapping". This is to avoid user's confusion towards Greek's final sigma and German's eszett. o Add the Section 4 "Open issues". o Modify the Section 8 "Security Considerations". o Modify the table format in the Appendix A. "Mapping type list each protocol". o Removed the Appendix B "Code points list for local case mapping". o Add the Appendix B "Local case mapping vs Case mapping". D.7. Changes since -06 o Removed the Section 4 "Open issues". o Change the title of the Appendix B "Local case mapping vs Case mapping" to "The reason why local case mapping is alternative to case mapping in PRECIS framework". o Add the Appendix C "Limitation to local case mapping". D.8. Changes since -07 o Modify the Section 1 "Introduction". o Modify the local case mapping's rule and target characters in the Section 2.3 "Local case mapping". o Modify the Section 3 "Order of operations". D.9. Changes since -08 o Updated the Unicode version 6.3 to 7.0 in this document. D.10. Changes since -09 o Modify the Section 1 "Introduction" to clarify to the discussion of string matching and the use of mappings from the SpecialCasing.txt. o Modify the Section 2.3 "Local case mapping" to clarify to the discussion of string matching and the use of mappings from the SpecialCasing.txt. o Modify the Appendix B "The reason why local case mapping is alternative to case mapping in PRECIS framework" to state the result of the case mapping in SpecialCasing.txt of eszett. o Clarify the Appendix C "Limitation to local case mapping". D.11. Changes since -10 o Modify the "Abstract" to clarify to sentences. o Modify the Section 1 "Introduction" to clarify to a sentence. o Modify the Section 2.2 "Special mapping" to add examples. o Modify the Section 2.3 "Local case mapping" to clarify to sentences. And add a note to explain the term "locale" in this document. o Modify the Section 3 "Order of operations" to clarify to sentences. o Correct a sentence in the Section 4 "Security Considerations". o Modify a sentence in the Section 6 "Acknowledgment". o Change the references from [I-D.ietf-precis-framework] to [RFC7564] in the Section 7 "Normative References". o Removed SASL and XMPP in the table of the Appendix A. "Mapping type list each protocol". o Modify the Appendix B "The reason why local case mapping is alternative to case mapping in PRECIS framework" to clarify to sentences. o Modify the Appendix C "Limitation to local case mapping" to clarify to sentences. D.12. Changes since -11 o Correct a few sentence in the Section 2.3 "Local case mapping" to address comments by the IESG review. o Removed citation part which includes "RECOMMENDED" (RFC 2119 word) in the Section 2.3 "Local case mapping" to avoid readers' confusion. o Modify the Section 4 "Security Considerations" to add a reference to RFC7564.during working group discussions. Authors' Addresses Yoshiro YONEYA JPRS Chiyoda First Bldg. East 13F 3-8-1 Nishi-Kanda Chiyoda-ku, Tokyo 101-0065 Japan Phone: +81 3 5215 8451 Email: yoshiro.yoneya@jprs.co.jp Takahiro Nemoto Keio University Graduate School of Media Design 4-1-1 Hiyoshi, Kohoku-ku Yokohama, Kanagawa 223-8526 Japan Phone: +81 45 564 2517 Email: t.nemo10@kmd.keio.ac.jp