• Re: Proposed Data Structure?

    From tenser@21:1/101 to stizzed on Wed Jun 3 07:36:58 2020
    On 02 Jun 2020 at 01:35p, stizzed pondered and said...

    Here is my first stab at opening conversation regarding data structure
    and format for a new BBSINFO standard. This example would follow the
    Fido standard of a flat file with one record per line, CR/LF terminated with comma delimiters and no spaces (using underscores instead). It
    would differ from the Fido format in that there would be data length limits.

    This conflates two things:

    1. A text serialization of some structured data.
    2. A binary format for a data structure representing that data.

    For (2), specific record layouts are an implementation detail.
    For a specification that is (presumably) implemented by many
    different systems, there's no need to specify them; they will
    vary by language and target system.

    For (1), if you're already making a departure from the existing
    nodelist format, then take the opportunity to use one of the
    many existing structured text formats. JSON and its variants,
    protocol buffers or a variant and their text serializations,
    even XML would have significant advantages over an ersatz format.
    In particular, they're all extensible and parsers exist for any
    number of languages and environments.

    In other words, what you really want to define isn't so much a
    file format as a schema.

    --- Mystic BBS v1.12 A46 2020/04/20 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From apam@21:1/126 to vorlon on Wed Jun 3 16:37:58 2020
    Make the lengh of the fields that are 30 char longer. If your going
    to list the website url for example, 30 characters will run out
    very quickly.

    I don't think an arbitary length should be set at all.

    Maybe a length is required for data structures in MPL - then I guess MPL
    will have to truncate long strings, but theres always going to be
    something that doesn't fit.

    But that is implementation specific, and not required if using a format
    like JSON, YAML, XML, etc.

    I think making a document that looks like a nodelist but isn't a node
    list is a really bad idea, it might make it easier to port code that
    reads a nodelist, but it makes it harder for any other purpose.

    Andrew


    --- MagickaBBS v0.15alpha (Linux/x86_64)
    * Origin: HappyLand - telnet://magickabbs.com:2023/ (21:1/126)
  • From Avon@21:1/101 to stizzed on Thu Jun 4 19:49:46 2020
    On 02 Jun 2020 at 01:35p, stizzed pondered and said...

    Team,

    Here is my first stab at opening conversation regarding data structure
    and format for a new BBSINFO standard. This example would follow the

    Thanks for putting this forward. :)

    I'm going to comment on the fields first as opposed to how they could be captured, their specs etc.

    Net : Integer; // BBS NET NUMBER
    Node : Integer; // BBS NODE NUMBER

    Have you considered Zone? I'm just wondering if it would be necessary to capture that for anything, granted there are a number of zones in use in FTN land and a BBS could be a member of several. Perhaps all the more reason to capture the info?

    BbsName : String[30]; // BBS NAME (My New BBS)
    Location : String[30]; // BBS LOCATION (City, State, Country
    SysopName : String[30]; // SYSOP NAME (Johnny SysOp)

    all things I would expect to see, mirrors the key info in a nodelist really.

    Phone : String[15]; // BBS PHONE NUMBER (888-555-1212)

    I guess, but phone is going the way of the dinosaur. I guess if this is all about trying to provide as much info on a system that *may* offer this connectivity then I agree keep it on the list.

    BbsTN : String[30]; // BBS TELNET URL W/PORT (bbs.do ain.tld:23)
    BbsSSH : String[30]; // BBS SSH URL W/PORT (bbs.domain.tld:2 )

    I wonder if there are other protocols worth capturing? Does for example TOR count?

    BbsWeb : String[30]; // BBS WEBSITE URL (www.bbs.domain.tld)

    Yep

    Nodes : String[3]; // TOTAL # OF NODES SUPPORTED (DIALUP & TELNET)

    Not of interest to me but could be to some I guess...

    Software : String[20]; // BBS SOFTWARE (Mystic v1.12 a45)

    Yep

    Delete : Boolean; // FLAG FOR DELETION BY DOWNSTREAM SYSTEMS

    I don't understand that one

    Verified : LongInt; // DATE INFO WAS VERIFIED

    I guess this speaks to some external checking system right?

    LognType : Byte; // LOGIN TYPE (1=realname/2=handle/3=choic )
    cType : Byte; // BBS TYPE (1=dialup/2=telnet/3=both

    If we have telnet and say phone number defined would we need BBS TYPE?

    Not sure if LOGIN TYPE is needed, but yeah can see how some would want to
    state that.

    How about something for Networks carried?

    Perhaps something for times available if it was say only running weeknights
    and weekends?

    Perhaps something to capture some info about the theme of the BBS or assign a number of tags to capture some high level categorization of the BBS

    Something to denote Country

    Something to capture languages available on the system e.g. English and
    Spanish echos

    Something that could (a bit like the verified tag) show days uptime / last heard from..

    Perhaps if the system could report number of callers per day something to capture that, calls per week, per month, year ... most active day, most inactive day of the week.

    Just some rando ideas :)

    A note on record lengths: It is advisable to limit record lengths (something Fido wanted to do but by the time they considered implementation the nodelist was t too large to be considered feasible). Remember, in many cases we are working with 80 column displays so field limits will be important to providing aesthetically pleasing display
    from applications.

    I think as to how the data could be stored, the format etc. I also thought of something like a JSON format that is extensible (and I'm not an active user
    of this data format) but my hunch is it would be more useful to contemporary coders as a way of getting data in/out of this file.

    To the suggestions of using DNS, yes I also see merit, but I would like to suggest we try to find a way where we don't depend solely on DNS as the solution if we were to apply something being worked on as a file format /
    data base to DNS.

    Good in my view to have more than one way to get at the data or modify it.

    I do like a decentralized approach and wonder aloud how correct data entry of records by those contributing could be for want of a better word policed such that duff or incomplete data was not accepted in any given data field?

    If we did use DNS records (and again I am showing my lack of knowledge here) could software
    1) scrape a nodelist
    2) parse that to DNS text etc records
    3) allow people via web admin form/portal to add of amend perhaps not the actual zone file records up front, but edit what the contents of some of the TXT fields would be using some kind of online form?

    That software could generate some agreed nightly JSON or similar file that could be peered between a number of systems like a daily master nodelist and used by devs as another source of data for applications to pull info from.

    I'm just trying to figure out how data however it is rendered and stored is best updated and by whom and how as time goes by?

    So many questions :)

    This is fun.

    --- Mystic BBS v1.12 A46 2020/04/20 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)