Synchronet and SBBSecho has always treated the to, from, and subject fields in FidoNet "Stored Messages" (*.msg files) and "Packed Messages" (those contained in type 2 packets) as null-terminated strings with a maximum *usable* length of
35 characters for the "to" and "from" and a maximum *usable* length of 71 characters for the "subject".

However, in reviewing FTS-1 (http://ftsc.org/docs/fts-0001.016) my/our interpretatoin may be wrong.

FTS-1 is ambiguous about whether or not the last character of these fields may be used or not. In other words, if a "to" or "from" name is exactly 36 characters, is it legal to use all 36 characters and *not* include a null terminator in a stored message? It is a fixed-length field after-all, so a terminator should not be needed if all 36 characters are used. Similarly, would
it be possible to use all 72 characters for a message subject? This would be consistent with how the "password" field in a packet header is stored (no null terminator included for full-length passwords).

"Packed Messages" use variable length header fields, so even full-length header
fields (e.g. a 36-character to or from name) would still require a null terminator. But the spec is not clear:

| subject |
~ max 72 bytes ~
| null terminated |

It's not clear if that "null" is *included* in the max 72 bytes, or not. :-(

How does *your* implementation handle these fields? What would happen if you received a Stored Message where byte 71 (the 72nd byte) of the "subject" was non-null? Or if you received a packet that included a 72-character subject followed by a null? Both of these conditions do not appear to violate FTS-1, but I'm not sure how other programmers have interpetted these specs over the years.

It seems wasteful to have critical bytes in a packet header that are *always* zero, so if we could agree that byte 71 (couting from 0) of a subject and byte 35 (again, counting from 0) of to/from names are *usable*, that would make these message/packet formats a little more sane.

But in any case, the spec (FTS-1) needs clarification: I can easily justify either interpration, which could lead to wildly-incompatible implementations of
FTN message/packet generating and parsing software.
--- SBBSecho 3.08-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Hello Rob!

18 Aug 19 17:37, you wrote to all:

Synchronet and SBBSecho has always treated the to, from, and subject fields in FidoNet "Stored Messages" (*.msg files) and "Packed
Messages" (those contained in type 2 packets) as null-terminated
strings with a maximum *usable* length of 35 characters for the "to"
and "from" and a maximum *usable* length of 71 characters for the "subject".

However, in reviewing FTS-1 (http://ftsc.org/docs/fts-0001.016) my/our interpretatoin may be wrong.

FTS-1 is ambiguous about whether or not the last character of these
fields may be used or not. In other words, if a "to" or "from" name is exactly 36 characters, is it legal to use all 36 characters and *not* include a null terminator in a stored message? It is a fixed-length
field after-all, so a terminator should not be needed if all 36
characters are used. Similarly, would it be possible to use all 72 characters for a message subject? This would be consistent with how
the "password" field in a packet header is stored (no null terminator included for full-length passwords).

"Packed Messages" use variable length header fields, so even
full-length header fields (e.g. a 36-character to or from name) would still require a null terminator. But the spec is not clear:

| subject |
~ max 72 bytes ~
| null terminated |

It's not clear if that "null" is *included* in the max 72 bytes, or
not. :-(

You raise a valid point there, Rob. The FTSC will have to look into that.

How does *your* implementation handle these fields? What would happen
if you received a Stored Message where byte 71 (the 72nd byte) of the "subject" was non-null? Or if you received a packet that included a 72-character subject followed by a null? Both of these conditions do
not appear to violate FTS-1, but I'm not sure how other programmers
have interpetted these specs over the years.

I will look into how the software I maintain (MBSE primarily, although MakeNL does generate Stored Messages as well) would react in those scenarios.

It seems wasteful to have critical bytes in a packet header that are *always* zero, so if we could agree that byte 71 (couting from 0) of a subject and byte 35 (again, counting from 0) of to/from names are *usable*, that would make these message/packet formats a little more
sane.

But in any case, the spec (FTS-1) needs clarification: I can easily justify either interpration, which could lead to wildly-incompatible implementations of FTN message/packet generating and parsing software.

I will forward this to the FTSC for discussion. IIRC, there are copyright issues involved with updating FTS-0001, so we may end up having to rewrite it entirely.

Regards,

Andrew

--- GoldED+/LNX 1.1.5-b20180707
* Origin: Phoenix BBS * phoenix.bnbbbs.net (1:320/219)

Re: Max subject length: 71 or 72 chars?
By: Andrew Leary to Rob Swindell on Mon Aug 19 2019 02:16 am

I will forward this to the FTSC for discussion. IIRC, there are copyright issues involved with updating FTS-0001, so we may end up having to rewrite it entirely.

Well that's long overdue anyway! :-)
--- SBBSecho 3.08-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Hello Rob!

18 Aug 19 17:37, you wrote to all:

Synchronet and SBBSecho has always treated the to, from, and subject fields in FidoNet "Stored Messages" (*.msg files) and "Packed
Messages" (those contained in type 2 packets) as null-terminated
strings with a maximum *usable* length of 35 characters for the "to"
and "from" and a maximum *usable* length of 71 characters for the "subject".

After discussion in the FTSC echo, it is the FTSC's position that this is the correct interpretation of FTS-0001.016. These fields must be null-terminated, and the length specified includes the required NUL (0x00) byte.

Andrew

--- GoldED+/LNX 1.1.5-b20180707
* Origin: Phoenix BBS * phoenix.bnbbbs.net (1:320/219)

Re: Max subject length: 71 or 72 chars?
By: Andrew Leary to Rob Swindell on Fri Aug 23 2019 08:09 am

Hello Rob!

18 Aug 19 17:37, you wrote to all:

Synchronet and SBBSecho has always treated the to, from, and subject fields in FidoNet "Stored Messages" (*.msg files) and "Packed
Messages" (those contained in type 2 packets) as null-terminated strings with a maximum *usable* length of 35 characters for the "to" and "from" and a maximum *usable* length of 71 characters for the "subject".

After discussion in the FTSC echo, it is the FTSC's position that this is the correct interpretation of FTS-0001.016. These fields must be null-terminated, and the length specified includes the required NUL (0x00) byte.

Cool. Let's document that position. :-)
--- SBBSecho 3.09-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Synchronet and SBBSecho has always treated the to, from, and subject
fields in FidoNet "Stored Messages" (*.msg files) and "Packed
Messages" (those contained in type 2 packets) as null-terminated
strings with a maximum *usable* length of 35 characters for the "to"
and "from" and a maximum *usable* length of 71 characters for the
"subject".

After discussion in the FTSC echo, it is the FTSC's position that this is
the correct interpretation of FTS-0001.016. These fields must be
null-terminated, and the length specified includes the required NUL (0x00)
byte.

Cool. Let's document that position. :-)

Please be aware that this "discussion" is nothing more than a single message by
a single person...

Ward
--- AfterShock/Android 1.6.8
* Origin: Baby-Glacier (2:292/854.1)

Good ${greeting_time}, Ward!

24 Aug 2019 11:38:00, you wrote to Rob Swindell:

After discussion in the FTSC echo, it is the FTSC's position that
this is the correct interpretation of FTS-0001.016. These fields
must be null-terminated, and the length specified includes the
required NUL (0x00) byte.

Cool. Let's document that position. :-)

Please be aware that this "discussion" is nothing more than a single message by a single person...

... with nobody been able to disprove that :-)


--
Alexey V. Vissarionov aka Gremlin from Kremlin
gremlin.ru!gremlin; +vii-cmiii-ccxxix-lxxix-xlii

... :wq!
--- /bin/vi
* Origin: http://openwall.com/Owl (2:5020/545)

Hello Alexey.

24 Aug 19 19:26, you wrote to Ward Dossche:

... with nobody been able to disprove that :-)

Your explanation of the issue was clear and concise. No further discussion was
needed.

Later,
Sean

--- GoldED/2 3.0.1
* Origin: Outpost BBS * bbs.outpostbbs.net:2304 (1:18/200)

On 2019 Aug 24 19:26:00, you wrote to Ward Dossche:

After discussion in the FTSC echo, it is the FTSC's position that
this is the correct interpretation of FTS-0001.016. These fields
must be null-terminated, and the length specified includes the
required NUL (0x00) byte.

Cool. Let's document that position. :-)

Please be aware that this "discussion" is nothing more than a single
message by a single person...

... with nobody been able to disprove that :-)

that it is (so far) a single message by one individual? why would anyone want to disprove that? ;)

)\/(ark

Once men turned their thinking over to machines in the hope that this would set
them free. But that only permitted other men with machines to enslave them.
... The vegetarian is almost as bad as a Communist.
---
* Origin: (1:3634/12.73)

On 2019 Aug 24 12:37:30, you wrote to Alexey Vissarionov:

... with nobody been able to disprove that :-)

Your explanation of the issue was clear and concise. No further

discussion

was needed.

agreed... the problem, as it were, stems from a non-coder having written the documentation in question... for its day, it made a lot of sense and there were
no ambiguities... these days, however, is another story...

)\/(ark

Once men turned their thinking over to machines in the hope that this would set
them free. But that only permitted other men with machines to enslave them.
... AuH2O! If you know what that means you ARE old!
---
* Origin: (1:3634/12.73)

Hello mark.

24 Aug 19 14:42, you wrote to me:

agreed... the problem, as it were, stems from a non-coder having
written the documentation in question... for its day, it made a lot of sense and there were no ambiguities... these days, however, is another story...

Yep. Works for me.

Later,
Sean

--- GoldED/2 3.0.1
* Origin: Outpost BBS * bbs.outpostbbs.net:2304 (1:18/200)

Alexey,

Please be aware that this "discussion" is nothing more than a single
message by a single person...

... with nobody been able to disprove that :-)

Nothing needed to be disproven as the spec is pretty straightforward. It's just
that native english speakers needed to be made aware.

After having struggled through IBM, Siemens and Fujitsu manuals for mainframes which are probably written by Pakistani people, this was easy.

My reaction was merely prompted by "one single message" being promoted to "discussion". Let's not create the impression the FTSC sweated a lot because of
this. You know this perfectly well.

Ward
--- AfterShock/Android 1.6.8
* Origin: Baby-Glacier (2:292/854.1)

Re: Max subject length: 71 or 72 chars?
By: Ward Dossche to Alexey Vissarionov on Sun Aug 25 2019 04:10 pm

Alexey,

Please be aware that this "discussion" is nothing more than a single
message by a single person...

... with nobody been able to disprove that :-)

Nothing needed to be disproven as the spec is pretty straightforward.

I disagree. I think it's ambiguous and incomplete, if not misleading.

It's just that native english speakers needed to be made aware.

Is FTS-1 available in another language, where it's possibly more complete? I'm not clear what "english speaking" has to do with it.

Here's what FTS-1 has to say about the message subject in Stored Messages in section B1:
+-----------------------+-----------------------+
72 48 | |
~ subject ~
| 72 bytes |
+-----------------------+-----------------------+
...
subject(72) (* see FileList below *)

No mention that a null-terminator is required.
No mention that the actual/usable number of characters in a message subject is in fact 71 characters (not 72).

Now, section C1 (Packed Message definition) is little better:

+-----------------------+-----------------------+
| subject |
~ max 72 bytes ~
| null terminated |
+-----------------------+-----------------------+
...
subject{72} (* Null terminated *)

But again, it should be made clear that the "72 bytes" here includes the required null terminator, so a message subject is in fact limited to a maxmimum
length of 71 characters.
--- SBBSecho 3.09-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Hallo mark!

24.08.2019 14:41, mark lewis schrieb an Alexey Vissarionov:

that it is (so far) a single message by one individual? why would
anyone want to disprove that? ;)

I checked the note, and agreed to .016 without any comment.
Any further comments to the existing .016 aren't necessary from my point of view.

)\/(ark

Bye/2 Torsten

... MAILBOX01 on OS2: up 3d 23h 31m load: 33 proc, 127 threads (tbupv1.1)
--- GoldED+ 1.1.5-19
* Origin: DatenBahn BBS Hamburg (2:240/5832)

Hello Rob.

25 Aug 19 14:53, you wrote to Ward Dossche:

I disagree. I think it's ambiguous and incomplete, if not misleading.

I'm in agreement with Andrew that it needs to be rewritten.

Later,
Sean

--- GoldED/2 3.0.1
* Origin: Outpost BBS * bbs.outpostbbs.net:2304 (1:18/200)

Hi Rob,

But again, it should be made clear that the "72 bytes" here includes the required null terminator, so a message subject is in fact limited to a maxmimum length of 71 characters.

From my point of view you were cherry-picking, probably involuntary, from sections B1 and C1. They're 2 different kinds of beast.

And do you also know where the "71" comes from ?

\%/@rd

--- D'Bridge 3.99
* Origin: Do not meddle in the affairs of wizards (2:292/854)

Re: Re: Max subject length: 71 or 72 chars?
By: Ward Dossche to Rob Swindell on Mon Aug 26 2019 01:53 am

Hi Rob,

But again, it should be made clear that the "72 bytes" here includes the required null terminator, so a message subject is in fact limited to a maxmimum length of 71 characters.

From my point of view you were cherry-picking, probably involuntary, from sections B1 and C1.

I'm pointing out the issues with the "subject" definition. Did you want me to quote the entire spec?

They're 2 different kinds of beast.

If you eliminate or ignore the "Stored Message" definition from the spec, then it would be less contradictory. I also think the Stored Message definition doesn't really belong in the same spec as the packet definition.

And do you also know where the "71" comes from ?

Well, I took 72 and subtracted 1 (for the required null byte). I assume the 72 came from 36 (the total length of the to and from fields) multiplied by 2. Is there some other significance of 71 that I'm not aware of?
--- SBBSecho 3.09-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Hi Rob,

On 2019-08-18 17:37:31, you wrote to All:

FTS-1 is ambiguous about whether or not the last character of these
fields may be used or not. In other words, if a "to" or "from" name is exactly 36 characters, is it legal to use all 36 characters and *not* include a null terminator in a stored message? It is a fixed-length
field after-all, so a terminator should not be needed if all 36
characters are used. Similarly, would it be possible to use all 72 characters for a message subject? This would be consistent with how
the "password" field in a packet header is stored (no null terminator included for full-length passwords).

"Packed Messages" use variable length header fields, so even full-length header fields (e.g. a 36-character to or from name) would still require a null terminator. But the spec is not clear:

| subject |
~ max 72 bytes ~
| null terminated |

It's not clear if that "null" is *included* in the max 72 bytes, or not. :-(

I think it is clear, and included in the 72 bytes.

How does *your* implementation handle these fields? What would happen
if you received a Stored Message where byte 71 (the 72nd byte) of the "subject" was non-null? Or if you received a packet that included a 72-character subject followed by a null? Both of these conditions do
not appear to violate FTS-1, but I'm not sure how other programmers
have interpetted these specs over the years.

I checked the fmail source: it reads a maximum of 72 bytes (untill and including a NULL) from the pkt file for the subject. It doesn't care if byte 71
isn't a NULL (when the maximum of 72 bytes are read), but it forces it to NULL before further processing. So effectively limiting the subject to 71 characters. If the byte after 72 subject characters would be NULL, it regards that as the 1st byte of the next field. So that would mean a message body of 0 characters.

Bye, Wilfred.

--- FMail-lnx64 2.1.0.18-B20170815
* Origin: FMail development HQ (2:280/464)

Re: Re: Max subject length: 71 or 72 chars?
By: Wilfred van Velzen to Rob Swindell on Mon Aug 26 2019 09:16 am

How does *your* implementation handle these fields? What would happen if you received a Stored Message where byte 71 (the 72nd byte) of the "subject" was non-null? Or if you received a packet that included a 72-character subject followed by a null? Both of these conditions do not appear to violate FTS-1, but I'm not sure how other programmers have interpetted these specs over the years.

I checked the fmail source: it reads a maximum of 72 bytes (untill and including a NULL) from the pkt file for the subject. It doesn't care if byte 71 isn't a NULL (when the maximum of 72 bytes are read), but it forces it to NULL before further processing. So effectively limiting the subject to 71 characters. If the byte after 72 subject characters would be NULL, it regards that as the 1st byte of the next field. So that would mean a message body of 0 characters.

Thanks for the detailed reply.

-Rob
--- SBBSecho 3.09-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Hi Rob,

I'm pointing out the issues with the "subject" definition. Did you want
me to quote the entire spec?

Oh dear ...

Something was clear for all the developers since 1990, there were a lot of them, and now suddenly it's an issue?

And do you also know where the "71" comes from ?

Well, I took 72 and subtracted 1 (for the required null byte). I assume
the 72 came from 36 (the total length of the to and from fields)
multiplied by 2. Is there some other significance of 71 that I'm not
aware of?

My IT experience starts somewhere in the very late 1960, punchcard heaven.

A punch card had 80 columns, 1-71 were used for coding statements, 72 was the continuation column if something didn't fit in 71 positions, then it continued on the next card, 73-80 contained the sequence number of that card in the whole
deck of punch cards.

It's not certain but my bet is the 71 comes from there, someone's knowledge/experience embedded in punchcard technology.

Just like the old length of the system name limited to 51 or 52 or something. It also finds its origin in programming techniques from days gone by ... I think.

\%/@rd

--- D'Bridge 3.99
* Origin: Do not meddle in the affairs of wizards (2:292/854)

Hey Ward!

It's not certain but my bet is the 71 comes from there, someone's knowledge/experience embedded in punchcard technology.

Possibly but from my past experience I believe it has to do with line lengths on tty's, line printers, and the such which were as a rule 80 characters wide.
The 'missing' 8 characters had to do with margins if I am not mistaken. As for
the null terminator that is a hangover from strings being terminated with nulls. I am unsure where that idea originates but I suspect pre-F77 FORTRAN.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.7(1)-release (aarch64-raspi3b+-linux-gnu)
* Origin: Little Mikey's CanadARM - Ladysmith BC, Canada (1:153/7001.2989)

Hallo Maurice!

As for the null terminator that is a hangover from strings being terminated with nulls. I am unsure where that idea originates
but I suspect pre-F77 FORTRAN.

According to Wikipedia it is a C thing but you're right about it being used as a string terminator which facillitates variable lengths for strings and has nothing to do with a fixed field length of 72 (or 71) bytes.

Het leven is goed,
Maurice

... Huil niet om mij, ik heb vi.
--- GNU bash, version 5.0.9(1)-release (x86_64-pc-linux-gnu)
* Origin: Little Mikey's EuroPoint - Ladysmith BC, Canada (2:280/464.113)

Re: Re: Max subject length: 71 or 72 chars?
By: Ward Dossche to Rob Swindell on Tue Aug 27 2019 05:01 pm

Hi Rob,

I'm pointing out the issues with the "subject" definition. Did you want me to quote the entire spec?

Oh dear ...

Something was clear for all the developers since 1990, there were a lot of them, and now suddenly it's an issue?

And do you also know where the "71" comes from ?

Well, I took 72 and subtracted 1 (for the required null byte). I assume the 72 came from 36 (the total length of the to and from fields) multiplied by 2. Is there some other significance of 71 that I'm not aware of?

My IT experience starts somewhere in the very late 1960, punchcard heaven.

A punch card had 80 columns, 1-71 were used for coding statements, 72 was th continuation column if something didn't fit in 71 positions, then it continu on the next card, 73-80 contained the sequence number of that card in the wh deck of punch cards.

It's not certain but my bet is the 71 comes from there, someone's knowledge/experience embedded in punchcard technology.

Just like the old length of the system name limited to 51 or 52 or something It also finds its origin in programming techniques from days gone by ... I think.

\%/@rd

FYI, it may be a replication of old ARPANET where the 'chat' was limited to 71 characters and a return. Yeah, I go back a ways.
xxcarol
--- SBBSecho 2.11-Win32
* Origin: SHENK'S EXPRESS telnet://shenks.synchro.net (1:275/100)