Hi guys,

I've been looking at this (because I wrote the code back in the mists of time).

What I'm sending as EOL characters with the QWK networking is chr(227). The spec seems to indicate that:

"For unclear reasons, new-lines sequences are normally represented by the character 0xE3 (227). In UTF-8 encoded messages, Synchronet uses ASCII 10 (LF)t o represent new-lines in QWK message bodies."

So... what should I do to make these avatar messages parse on systems other than Vert (until everyone updates to a new version)? I think I'm following the spec and the spec says that Synchronet converts to LF, not CR/LF.

I could change it to send CR/LF in the QWk packet but that is against the specification.

Thanks,

Mark


---
* TARDIS BBS - Home of QUARKware * telnet bbs.cortex-media.info
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: More Questions
By: the doctor to Digital Man / echicken on Thu Feb 24 2022 03:44 pm

Hi guys,

I've been looking at this (because I wrote the code back in the mists of time).

What I'm sending as EOL characters with the QWK networking is chr(227). The spec seems to indicate that:

"For unclear reasons, new-lines sequences are normally represented by the character 0xE3 (227). In UTF-8 encoded messages, Synchronet uses ASCII 10 (LF)t o represent new-lines in QWK message bodies."

So... what should I do to make these avatar messages parse on systems other than Vert (until everyone updates to a new version)? I think I'm following the spec and the spec says that Synchronet converts to LF, not CR/LF.

I could change it to send CR/LF in the QWk packet but that is against the specification.

227 is right (for legacy/non-UTF-8 messages). Synchronet expands 227 (0xE3) to CRLF upon import, e.g.

000000f0 0d 0a 57 68 61 74 20 49 27 6d 20 73 65 6e 64 69 |..What I'm sendi| 00000100 6e 67 20 61 73 20 45 4f 4c 20 63 68 61 72 61 63 |ng as EOL charac| 00000110 74 65 72 73 20 77 69 74 68 20 74 68 65 20 51 57 |ters with the QW| 00000120 4b 20 6e 65 74 77 6f 72 6b 69 6e 67 20 69 73 20 |K networking is | 00000130 63 68 72 28 32 32 37 29 2e 20 20 54 68 65 20 0d |chr(227). The .| 00000140 0a 73 70 65 63 20 73 65 65 6d 73 20 74 6f 20 69 |.spec seems to i| 00000150 6e 64 69 63 61 74 65 20 74 68 61 74 3a 0d 0a 0d |ndicate that:...| 00000160 0a 22 46 6f 72 20 75 6e 63 6c 65 61 72 20 72 65 |."For unclear re|

But for whatever reason, your Avatar/syncdata messages were being imported differently:

000000b0 30 0a 6a 73 6f 6e 2d 62 65 67 69 6e 0a 7b 0a 20 |0.json-begin.{. | 000000c0 20 22 41 51 59 46 6f 48 59 55 47 32 75 65 69 47 | "AQYFoHYUG2ueiG| 000000d0 77 50 6f 52 51 4a 71 67 62 5a 6d 74 69 41 65 31 |wPoRQJqgbZmtiAe1| 000000e0 49 53 74 67 50 41 44 37 35 41 75 79 71 52 64 58 |IStgPAD75AuyqRdX| 000000f0 54 56 41 41 30 43 36 75 6c 4b 33 75 58 50 5a 4e |TVAA0C6ulK3uXPZN| 00000100 75 79 77 64 63 57 50 6a 6b 6c 69 6f 41 52 71 69 |uywdcWPjklioARqi| 00000110 56 45 43 75 56 30 57 79 6f 51 41 3d 3d 3d 22 3a |VECuV0WyoQA===":| 00000120 20 5b 0a 20 20 20 20 22 53 59 53 4f 50 22 0a 20 | [. "SYSOP". | 00000130 20 5d 2c 0a 20 20 22 41 51 41 46 48 32 43 67 42 | ],. "AQAFH2CgB| 00000140 33 4f 4f 61 53 76 46 69 42 4e 54 49 47 33 49 61 |3OOaSvFiBNTIG3Ia| 00000150 68 32 65 41 52 74 32 7a 49 77 76 4e 4a 4b 68 73 |h2eARt2zIwvNJKhs| 00000160 79 69 68 33 30 33 6b 59 58 69 59 57 58 5a 39 54 |yih303kYXiYWXZ9T| 00000170 38 69 41 22 3a 20 5b 0a 20 20 20 20 22 74 68 65 |8iA": [. "the| 00000180 20 64 6f 63 74 6f 72 22 0a 20 20 5d 0a 7d 0a 6a | doctor". ].}.j| 00000190 73 6f 6e 2d 65 6e 64 0d 0a 0a 0d 0a 0d 0a 0d 0a |son-end.........|

(LF-terminated lines, not CRLF).
--
digital man (rob)

Sling Blade quote #18:
Karl Childers: Some folks call it Hell, I call it Hades.
Norco, CA WX: 58.8øF, 22.0% humidity, 8 mph SE wind, 0.00 inches rain/24hrs
--- SBBSecho 3.14-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

On 2/24/22 08:44, the doctor wrote:

What I'm sending as EOL characters with the QWK networking is chr(227). The spec seems to indicate that:

"For unclear reasons, new-lines sequences are normally represented by the character 0xE3 (227). In UTF-8 encoded messages, Synchronet uses ASCII 10 (LF)t o represent new-lines in QWK message bodies."

So... what should I do to make these avatar messages parse on systems other than Vert (until everyone updates to a new version)? I think I'm following the spec and the spec says that Synchronet converts to LF, not CR/LF.

I could change it to send CR/LF in the QWk packet but that is against the specification.

I can't speak for how Synchronet is handling this...

I have been looking into this as well, planning on cataloging the
information into a couple services as well. My plan was to look for "json-begin" and "json-end" in the message and plucking what's in between.

Per the JSON spec, it *should* be UTF8 encoded, and any white-space
characters outside quoted strings should be ignored. Since you can
escape any character outside the 7-bit ASCII range (as well as quotes)
per JS encoding (\x## \u#### \###), I suggest doing so for any
interchange messages that include higher characters (and control chars),
and (re)encoding any non-utf8 character sets to their Unicode
corresponding values.

As to message bodies, I think that terminal based message conventions is
the combination of CRLF (\r\n), since \r will move the cursor to the
beginning of the line and \n will move the cursor down one line. (IIRC)

That's just my own opinionated take on this one, and what I've been
doing so far.
--
Michael J. Ryan - tracker1@roughneckbbs.com
---
ï¿­ Synchronet ï¿­ Roughneck BBS - roughneckbbs.com
--- SBBSecho 3.15-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

--- TRACKER1 wrote ---
On 2/24/22 08:44, the doctor wrote:
I can't speak for how Synchronet is handling this...

I have been looking into this as well, planning on cataloging the information into a couple services as well. My plan was to look for "json-begin" and "json-end" in the message and plucking what's in
between.

Per the JSON spec, it *should* be UTF8 encoded, and any white-space characters outside quoted strings should be ignored. Since you can
escape any character outside the 7-bit ASCII range (as well as quotes)
per JS encoding (\x## \u#### \###), I suggest doing so for any
interchange messages that include higher characters (and control chars),
and (re)encoding any non-utf8 character sets to their Unicode
corresponding values.

As to message bodies, I think that terminal based message conventions is
the combination of CRLF (\r\n), since \r will move the cursor to the beginning of the line and \n will move the cursor down one line. (IIRC)

That's just my own opinionated take on this one, and what I've been
doing so far.

Thanks for the information. I will trying doing what you suggest for
unicode characters. I've more or less got it working now, it was really
a problem caused by the more dusty bits of code (that i wrote like 15 or
more years ago) and me not remember what I did or why. (:


---
"No matter where you go, there you are..."


---
* TARDIS BBS - Home of QUARKware * telnet bbs.cortex-media.info
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)