• Clearing Houz

    From apam@21:1/182 to deon on Fri Oct 7 16:05:01 2022
    Hey

    I was just wondering how you store messages in Clearing Houz? I think
    from memory you use couchdb or cockroach db or something?

    Is that using a similar format to JSON?

    I'm going to experiment with MongoDB on my rust BBS, and wondered if
    there was a way to transmit messages in JSON format.

    I don't really want to implement FTN, as this BBS is for experimenting,
    but I thought it might be fun to experiment with building a new sort of "network".

    I was wondering if something like clearing houz would be able to easily
    gate messages between FTN and whatever?

    Anyway, I've not thought this through very much yet, just tossing out
    ideas of things that might be fun to do (My son is out for the day with
    his mum visiting poppy - so I have some free time ;P)

    Andrew


    --- Talisman v0.43-dev (Windows/x64)
    * Origin: Smuggler's Cove - Private BBS (21:1/182)
  • From deon@21:2/116 to apam on Fri Oct 7 17:44:40 2022
    Re: Clearing Houz
    By: apam to deon on Fri Oct 07 2022 04:05 pm

    Howdy,

    I was just wondering how you store messages in Clearing Houz? I think
    from memory you use couchdb or cockroach db or something?

    So I started Clearing Houz using mongodb only because, I think like you, I wanted to check it out. For me it was problematic because it doesnt enforce a schema (it's benefit) and I found I was spending too much time trying to fix problems, only to find I had a typo in the schema or query here and there. (And probably because I dont know it natively like SQL, so learning how to query it outside of clrghouz was a learning curve I was too impatient to learn.)

    I also wanted the replication - one of the designs I had for Clearing Houz, so that nodes could drop mail off (and pick it up) from any place. I think mongo can do, but I had to reasearch how to do it.

    So I did switch to CockroachDB - which is SQL and uses the postgres protocol (and in doing so, I discovered an emabarrising amount of mistakes that I had made :) It provided the geo replication that I was wanting out of the box and it is very easy to use (probably because I know SQL better).

    (Meatlotion provided a couple of nodes for cockroach - so when you were hitting the website (and hub 3 was tossing mail) it was actually updating the DB that was spread between AU and UK.)

    I've put Clearing Houz to sleep for a bit, mainly because I'm busy with other things and because I need to address a latency issue that I thought was making the web experience not great. (I wanted to enable sysops to configure, sign up and almost be able to self-diagnose why their mail wasnt flowing via it). So I may end up ditching cockroachdb if I cant fix that... :(

    Is that using a similar format to JSON?

    Mongo does return (and I guess does accept) JSON data when querying and storing. I was actually using via an SQL like interface (Clearing Houz is built on laravel - and it has a query interface called eloquent that abstracts the database store).

    I'm going to experiment with MongoDB on my rust BBS, and wondered if
    there was a way to transmit messages in JSON format.

    I don't really want to implement FTN, as this BBS is for experimenting,
    but I thought it might be fun to experiment with building a new sort of "network".

    I was wondering if something like clearing houz would be able to easily
    gate messages between FTN and whatever?

    That was another one of my goals with Clearing Houz - given that I had messages in "a database", writing different "packet" formats should be very easy - and I had thought of enabling an API to get/put echomail using a POST with a JSON payload. In my case I'd consume the JSON payload before it got to the DB, but in terms of sharing data around, I do think its a good any easy format to use. (I do a lot of APIs and JSON payloads.)

    Anyway, I've not thought this through very much yet, just tossing out
    ideas of things that might be fun to do (My son is out for the day with
    his mum visiting poppy - so I have some free time ;P)

    Awesome, treasures those moments, they seem to get smaller every day !

    If you want to play with something, let me know, and I'll turn back on clearing houz - happy to set something up if you need another target...


    ...лоеп
    --- SBBSecho 3.15-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From apam@21:1/182 to deon on Fri Oct 7 17:14:27 2022
    If you want to play with something, let me know, and I'll turn back on clearing houz - happy to set something up if you need another
    target...

    Thanks for the reply, I didn't realize you'd turned clearing houz off, I understand though, only so many hours in the day. I did think it was a
    very cool idea though.

    Right now, this is just random ideas in my head.. not sure if they'll work
    or not. I'll let you know if I come up with something more tangible.

    Andrew


    --- Talisman v0.43-dev (Windows/x64)
    * Origin: Smuggler's Cove - Private BBS (21:1/182)
  • From Oli@21:3/102 to apam on Fri Oct 7 08:53:33 2022
    apam wrote (2022-10-07):

    Hey

    I was just wondering how you store messages in Clearing Houz? I think
    from memory you use couchdb or cockroach db or something?

    Is that using a similar format to JSON?

    I haven't used couchdb for quite some time, only for trivial stuff and never in a cluster. Replication is easy to setup and it's all JSON* and Javascript.

    *fun fact: when couchdb was alpha 0.something and they used XML as their document format, I proposed they should support or switch to JSON. Nowadays I don't like JSON very much.


    вЃ‚

    ---
    * Origin: War is Peace. Freedom is Slavery. Ignorance is Strength. (21:3/102)
  • From Avon@21:1/101 to apam on Fri Oct 7 21:44:47 2022
    On 07 Oct 2022 at 04:05p, apam pondered and said...

    I don't really want to implement FTN, as this BBS is for experimenting, but I thought it might be fun to experiment with building a new sort of "network".

    If you do build something and would like to test stuff I would be keen to help.

    Kerr Avon [Blake's 7] 'I'm not expendable, I'm not stupid and I'm not going' avon[at]bbs.nz | bbs.nz | fsxnet.nz

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From apam@21:1/182 to Avon on Fri Oct 7 22:15:23 2022
    If you do build something and would like to test stuff I would be keen
    to help.

    Sure, I'll let you know if I come up with something.

    I'm not very inventive.

    I was playing with MongoDB, but am thinking it might be simpler to use Postgresql instead. It's not really a network though.

    The idea of what I want to do, is a central repository of messages,
    stored in a database.

    BBSes query the database via an API, it would be something like:

    https://some.host/getmsg/XXXXXX

    where XXXXXX is the last query id.

    the server responds with a json array of messages with an id higher than
    the last query id.

    to post a message it would send a POST query with a json message (and an
    api key for the bbs)

    each message has a random uuid so the bbs knows not to import messages
    that it exported

    oh well screaming baby

    andrew


    --- Talisman v0.43-dev (Windows/x64)
    * Origin: Smuggler's Cove - Private BBS (21:1/182)
  • From deon@21:2/116 to apam on Wed Nov 30 16:32:01 2022
    Re: Clearing Houz
    By: apam to deon on Fri Oct 07 2022 05:14 pm

    Hey Apam,

    If you want to play with something, let me know, and I'll turn back on clearing houz - happy to set something up if you need another
    target...

    Thanks for the reply, I didn't realize you'd turned clearing houz off, I understand though, only so many hours in the day. I did think it was a
    very cool idea though.

    Right now, this is just random ideas in my head.. not sure if they'll work or not. I'll let you know if I come up with something more tangible.

    Not sure if you are still playing with the idea of a URL / json kind of packet for data - but if you wanted to try something, then I'm happy to work on it with you.

    I've had a bit of energy of late (and time) to work a bit more on clrghouz - and I'm really hoping to get it in a state where it could be Hub 3.

    In parallel, it would be easy for me to create a URL endpoint that provides and accepts mail in another format (eg: json).


    ...лоеп
    --- SBBSecho 3.15-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)
  • From apam@21:1/182 to deon on Wed Nov 30 23:00:02 2022
    Not sure if you are still playing with the idea of a URL / json kind
    of packet for data - but if you wanted to try something, then I'm
    happy to work on it with you.

    Not really. I'd like to do something, but not much motivation at present.

    Cool to see you're working on it again. I thought it was a really neat
    project.

    Andrew


    --- Talisman v0.46-dev (Windows/x64)
    * Origin: Smuggler's Cove - Private BBS (21:1/182)
  • From deon@21:2/116 to apam on Thu Dec 1 00:22:36 2022
    Re: Clearing Houz
    By: apam to deon on Wed Nov 30 2022 11:00 pm

    Not really. I'd like to do something, but not much motivation at present.

    Fair enough - when you get your motivation back let me know...

    Cool to see you're working on it again. I thought it was a really neat project.

    Thanks :)


    ...лоеп
    --- SBBSecho 3.15-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)