XML 1.1 Spec Hits Some Snags 259
oever writes "News.com reports that the new XML 1.1 specification defines a new newline character, making it incompatible with the 1.0 specifiation. Apparently, IBM has been pushing the new character to avoid having to modify their software, thereby invalidating everybody else's XML software."
Read the Unicode spec.... (Score:5, Informative)
So? (Score:5, Informative)
1.1 : To simplify the tasks of applications, the characters passed to an application by the XML processor must be as if the XML processor normalized all line breaks in external parsed entities (including the document entity) on input, before parsing, by translating all of the following to a single #xA character:
I don't get it, whats the problem here? Surely the 1.1 spec simply extends the available EOL characters. It certainly doesn't remove any existing characters that are present in the 1.0 spec. How does it break backwards compatability?
What about poor old Acorn users? (Score:3, Informative)
Re:So? (Score:0, Informative)
Re:So? (Score:5, Informative)
Re:So? (Score:1, Informative)
Re:So? (Score:5, Informative)
So unless you are using a non-Unicode, non-ISO-Latin encoding there are no printable characters in that range, and if you're using another character you will need to remap the characters before considering any of the rules in the XML spec anyway, since those rules refer to the unicode codepoints.
Complain (Score:3, Informative)
Re:So? (Score:5, Informative)
Chris Beckenbach
XML 1.1 - Problem? (Score:5, Informative)
Full Details (Score:5, Informative)
Please read that before making uninformed comments - news.com isn't where you'll find technical information about this problem.
Re:Read the Unicode spec.... (Score:5, Informative)
2029 - paragraph seperator
2028 - line seperator
000D - CR
000A - LF
0085 - NEL (Next Line)
Any of these could be interpeted as the end of a logical line.
The proposed change (Score:4, Informative)
here [w3.org] is a summary of just the proposed change.
It seems to comply with unicode just fine, I dont see what the controversy is really.
For those missing the problem, also à (Score:2, Informative)
First off, unicode 85H is NEXT LINE; ASCII 85H is ellipses. unicode 2028H is LINE SEPARATOR. AH and DH are the infamous CR/LF which annoy MS/UNIX text convertors.
Anyways, it sounds like a problem comes up here:
must be as if the XML processor normalized all line breaks in external parsed entities (including the document entity) on input, before parsing,
That seems to be the sticking point there.
I suppose the charset is specified in the document, but then again I'm not sure how literally they intend implementors to take the phrase "before parsing", since getting at the charset description involves some degree of parsing the document
Re:Read the Unicode spec.... (Score:5, Informative)
Re:how hard is it to do parsing? (Score:3, Informative)
Re:For those missing the problem, also à (Score:5, Informative)
The W3C's XML 1.0 Recommendation was first issued in 1998, and despite the issuance of many errata culminating in a Second Edition of 2000, has remained (by intention) unchanged with respect to what is well-formed XML and what is not. This stability has been extremely useful for interoperability. However, the Unicode Standard on which XML 1.0 relies for character specifications has not remained static, evolving from version 2.0 to version 3.1 and beyond. Characters not present in Unicode 2.0 may already be used in XML 1.0 character data. However, they are not allowed in XML names such as element type names, attribute names, enumerated attribute values, processing instruction targets, and so on. In addition, some characters that should have been permitted in XML names were not, due to oversights and inconsistencies in Unicode 2.0
So XML 1.0 used Unicode 2.0, but not properly. XML 1.1 fixes that, and defines that all Unicode 3.2 byte pairs are now valid when used in an XML document. As part of this change, XML 1.1 also correctly allows the use of the Unicode 0x0085 NEL character as an EOL marker, which is totally compliant and consistent with the Unicode 3.2 specification.
In other words, if you're using any character encoding other than Unicode 3.2, your XML document isn't compliant with XML 1.1 and you shouldn't ever expect ISO-Latin 0x85 to be displayed as an ellipses.
Re:So? (Score:4, Informative)
30 78 38 35 20 69 73 20 e0 20 20 28 61 20 67 72 : 0x85 is à (a gr
61 76 65 29 2e 20 20 53 6f 20 65 76 65 72 79 6f : ave). So everyo
6e 65 20 69 6e 20 46 72 61 6e 63 65 3f 0a 09 09 : ne in France?...
Re:What? No newline? (Score:5, Informative)
Using the paragraph tags as large linebreaks is a very bad habit from the bad old days of the web. Please head to W3C [w3.org] and study the recent standards, and validate your documents before publishing (using a validator [w3.org], not a browser).
Ohh... And this is actually an issue about XML 1.1 unicode support, so worrying about HTML is quite premature (XHTML is still XML 1.0, and will remain so untill XML 1.1 becomes a standard (or recommendation in W3C-speak).
Re:New Newline Character? (Score:5, Informative)
Re:Read the Unicode spec.... (Score:2, Informative)
no. it's fine. (Score:4, Informative)
No. XML is defined in terms of Unicode, but XML files can be stored in any encoding with a known mapping to Unicode.
Most XML files these days are using iso-8859-1 or UTF-8, both of which manage fine in vi/pico/gedit.
chars 32-127 are identical in ASCII and Unicode, and iso-8859-1 is exactly identical to the bottom 8-bits worth of Unicode.
Also, UTF-8 is an encoding of the full Unicode range that is backwards-compatible with 7-bit ASCII.
in any case, note the entry for LF in your parent post -- 000A (hex) = 10 (decimal).
Re:Read the Unicode spec.... (Score:5, Informative)
What it *does* mean is that editors on other systems than Unix are able to edit XML files. It means I can create an XML file in DOS 'edit' which uses \r\n, or on a mac with an editor that might use \r, or on (apparently) an IBM system where the standard text editors use \u85.
This is absolutely essential. It does however mean that in order to support *all* XML files, you need to recognise *all* of those line endings. As always, its easier to support a subset, but harder to support everything. However the fact that existing software works at all is very important, so I think they're moving in the right direction.
Re:What do they mean, "XML 1.0 chokes"? (Score:2, Informative)
Many XML applications ignore whitespace (after parsing). XML parsers are prohibited from deleting any whitespace that might be part of the data in a document. The xml:space attribute allows a document to indicate places where the author or encoder encourages normalization of space in some way.
This is all clearly explained in the standard itself (W3c XML pages [w3.org]).
Re:Read the Unicode spec.... (Score:5, Informative)
There are no 2-byte Unicode characters, only encodings (such as UTF-16) which use two or more bytes to represent each character. Some Unicode characters, those not in the Basic Multilingual Plane (BMP), require more than two bytes to represent in UTF-16.
And 7-bit ASCII is a strict subset of UTF-8 encoding. UTF-8 encodes each character to one or more bytes, with characters up to 127 defined the same as in ASCII. If your text is strict 7-bit ASCII, it is also a UTF-8 file.
You could also use UTF-32 (UCS-4), which represents each character as 4 bytes, but that is overkill for most applications.
The main problem with multibyte encodings such as Shift-JIS and Big 5 was lead-byte detection: you couldn't jump into the middle of a string and determine if you were looking at the only, first, or second byte of a character. You had to start parsing at the beginning of the string in order to synchronize your character detection. Unicode has done away with this by strictly defining the lead byte ranges in such a way that there is never any ambiguity.
Re:Read the Unicode spec.... (Score:3, Informative)
Re:So? (Score:1, Informative)
But, unfortunately, the very commonly used Windows codepage 1252 puts printable glyphs in that code point range. Rather, recent versions of CP 1252 (not all versions of CP1252 are the same) have put things like trademark signs [dkuug.dk] in that character codepoint range. Keep in mind that Microsoft still dominates the computer software industry more so than IBM and their attempts to keep things open and standards body approved.
Re:One tiny little update ??? (Score:5, Informative)
Problem areas include:
* Processing XML documents or DTDs generated on OS/390 systems, with XML 1.0 compliant parsers.
* Processing XML documents or DTDs, using native OS/390 system tools.
* Processing XML documents or DTDs retrieved from OS/390 database or file systems, in non-OS/390 environments.
XML documents that contain [NEL] characters are declared invalid or not well-formed by XML 1.0 compliant parsers.
Essentially 'just fix the software' involves operating system-level changes as well as possibly changes to most software that interprets NEL characters on that OS. As it stands, they're going to have a problem anyway, and it's probably best to simply add the change to the XML standard to fix what was essentially an osmission in the 1.0 standard.
Re:One tiny little update ??? (Score:3, Informative)
Re:Full Details (Score:3, Informative)
Or specify iso-8859-1 as their encoding in the XML prologue, which they were supposed to do in XML 1.0 anyway.
depends on the character set (Score:3, Informative)
iso-8859-1:
NEL (from EBCDIC) doesn't exist in iso-8859-1; what character gets used instead is up to the discretion of the tool doing the conversion. Since Unicode defines 0x0085 as a whitespace character, the tool could know to substitute another if that was desired.
UTF-8:
It's a non-lossy encoding of Unicode. All characters above 0x007f get stored as multi-byte sequences. 0x0085 becomes 0xc2 0x85.
UTF-8 (Score:3, Informative)
Some clarifcation (Score:3, Informative)
Re:What? No newline? (Score:4, Informative)
This is not correct. The P tag ALWAYS had a closing tag, but it was not REQUIRED. Here's the P tag section for HTML 2.0:
http://www.w3.org/MarkUp/html-spec/L2index.html
was still allowed, but all it did was start a new paragraph.
Actually, it just ends the current paragraph. You can't use the closing tag without the beginning tag and still have well-formed HTML. Since it wasn't required, though, there are probably many different methods of handling it as far as HTML parsers/browsers are concerned. The only reason it would start a new paragraph is because it designates the end of the current paragraph, and another paragraph is what typically follows the end of a paragraph within a document.