• linux multi-port server design in c/c++

    From noloader@gmail.com@1:0/0 to All on Tue Nov 26 06:39:56 2013
    Hi All,

    Is anyone aware of a discussion or treatment of a multi-port threaded serve=
    r on modern Linux boxes written in c/c++? Multi-port would mean, for exampl=
    e, ports 80 and 443.

    Contemporary Linux and Unix post-date W. Richard Steven's Advanced Programm= ing in the Unix Environment and Unix Network Programming, so my bibles are = not helpful. Googling is returning the Comp Sci 101 examples with a single = server/single port.

    Specifically, I'm interested in discussions of:

    (1) multiple processes servicing a single port and thread the accept()
    (2) single process with multiple threads for each port and thread the accep= t()
    (3) the most efficient way to wait and handle connections
    (4) how to safely shutdown a multi-port, multi-threaded server

    I'm fairly certain fork/exec is no longer recommended, especially since cri= tical libraries like OpenSSL are not fork-safe in a multi-threaded environm= ent.

    I also think item (3) can be handled with libevent, but I'm also interested=
    in a proper discussion in this setting so I can avoid the dependency. (I t=
    ry to minimize or eliminate dependencies).

    Finally, my apologies if this is not a programming group. Google's Usenet i= nterface simply sucks. We can't navigate group hierarchies any longer under=
    the new and improved design. And searching for "comp.os.linux groups" retu= rned one result of comp.os.linux.misc.

    Thanks in advance.

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From David Brown@1:0/0 to All on Tue Nov 26 08:48:45 2013
    On 26/11/13 07:39, noloader@gmail.com wrote:
    Hi All,

    Is anyone aware of a discussion or treatment of a multi-port threaded
    server on modern Linux boxes written in c/c++? Multi-port would mean,
    for example, ports 80 and 443.

    Contemporary Linux and Unix post-date W. Richard Steven's Advanced Programming in the Unix Environment and Unix Network Programming, so
    my bibles are not helpful. Googling is returning the Comp Sci 101
    examples with a single server/single port.

    Specifically, I'm interested in discussions of:

    (1) multiple processes servicing a single port and thread the
    accept() (2) single process with multiple threads for each port and
    thread the accept() (3) the most efficient way to wait and handle
    connections (4) how to safely shutdown a multi-port, multi-threaded
    server

    I'm fairly certain fork/exec is no longer recommended, especially
    since critical libraries like OpenSSL are not fork-safe in a
    multi-threaded environment.

    I also think item (3) can be handled with libevent, but I'm also
    interested in a proper discussion in this setting so I can avoid the dependency. (I try to minimize or eliminate dependencies).

    Finally, my apologies if this is not a programming group. Google's
    Usenet interface simply sucks. We can't navigate group hierarchies
    any longer under the new and improved design. And searching for "comp.os.linux groups" returned one result of comp.os.linux.misc.

    Thanks in advance.


    This is not really a programming group - but you might get some ideas
    here. I don't know what the ideal group would be - comp.lang.c and comp.lang.c++ are more about the language than applications.

    There is no such language as "C/C++". A server like this would be
    structured totally differently in C and in C++. Don't mix these up -
    pick one language.

    Also, is there a particular reason to pick C or C++ ? There are lots of
    other languages suitable for server design, many with frameworks that
    will give you most of what you want here without any effort. Python
    with the "twisted" framework springs to mind (since I've used it myself)
    as being very efficient at handling multiple ports with multiple
    connections, and it takes care of all the details of resources. There
    are many other frameworks for many other languages which could make the
    job a lot easier.




    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From Chris Davies@110:110/2002 to All on Tue Nov 26 17:03:26 2013
    Reply-To: chris@roaima.co.uk

    noloader@gmail.com wrote:
    Is anyone aware of a discussion or treatment of a multi-port threaded
    server on modern Linux boxes written in c/c++? Multi-port would mean,
    for example, ports 80 and 443.

    Google offers many seemingly-relevant results for a search of
    "multithreaded socket select".


    (1) multiple processes servicing a single port and thread the accept()
    (2) single process with multiple threads for each port and thread the
    accept()
    (3) the most efficient way to wait and handle connections
    (4) how to safely shutdown a multi-port, multi-threaded server

    There may well be discussions on these if you throw the keyword "apache"
    in to the mix.

    Chris

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: Roaima. Harrogate, North Yorkshire, UK (110:110/2002@linuxnet)
  • From noloader@gmail.com@1:0/0 to All on Tue Nov 26 17:33:40 2013
    On Tuesday, November 26, 2013 12:03:26 PM UTC-5, Chris Davies wrote:
    noloader@gmail.com wrote:

    Is anyone aware of a discussion or treatment of a multi-port threaded

    server on modern Linux boxes written in c/c++? Multi-port would mean,

    for example, ports 80 and 443.



    Google offers many seemingly-relevant results for a search of

    "multithreaded socket select".
    Thanks Chris. Try adding "multi-port" (or one of its derivatives).

    (1) multiple processes servicing a single port and thread the accept()

    (2) single process with multiple threads for each port and thread the
    accept()

    (3) the most efficient way to wait and handle connections

    (4) how to safely shutdown a multi-port, multi-threaded server

    There may well be discussions on these if you throw the keyword "apache"

    in to the mix.
    Yeah, I'm trying to avoid studying Apache source code.

    Jeff

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From noloader@gmail.com@1:0/0 to All on Tue Nov 26 17:47:57 2013
    On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:
    On 26/11/13 07:39, noloader@gmail.com wrote:
    =20
    Hi All,
    =20
    =20
    =20
    Is anyone aware of a discussion or treatment of a multi-port threaded
    =20
    server on modern Linux boxes written in c/c++? Multi-port would mean,
    =20
    for example, ports 80 and 443.
    =20
    =20
    =20
    Contemporary Linux and Unix post-date W. Richard Steven's Advanced
    =20
    Programming in the Unix Environment and Unix Network Programming, so
    =20
    my bibles are not helpful. Googling is returning the Comp Sci 101
    =20
    examples with a single server/single port.
    =20
    =20
    =20
    Specifically, I'm interested in discussions of:
    =20
    =20
    =20
    (1) multiple processes servicing a single port and thread the
    =20
    accept() (2) single process with multiple threads for each port and
    =20
    thread the accept() (3) the most efficient way to wait and handle
    =20
    connections (4) how to safely shutdown a multi-port, multi-threaded
    =20
    server
    =20
    =20
    =20
    I'm fairly certain fork/exec is no longer recommended, especially
    =20
    since critical libraries like OpenSSL are not fork-safe in a
    =20
    multi-threaded environment.
    =20
    =20
    =20
    I also think item (3) can be handled with libevent, but I'm also
    =20
    interested in a proper discussion in this setting so I can avoid the
    =20
    dependency. (I try to minimize or eliminate dependencies).
    =20
    =20
    =20
    Finally, my apologies if this is not a programming group. Google's
    =20
    Usenet interface simply sucks. We can't navigate group hierarchies
    =20
    any longer under the new and improved design. And searching for
    =20
    "comp.os.linux groups" returned one result of comp.os.linux.misc.
    =20
    =20
    =20
    Thanks in advance.
    =20
    =20
    =20
    =20
    =20
    This is not really a programming group - but you might get some ideas
    =20
    here. I don't know what the ideal group would be - comp.lang.c and
    =20
    comp.lang.c++ are more about the language than applications.
    Oh, sorry about that.

    I'll try another group or Stack Overflow.

    Also, is there a particular reason to pick C or C++ ?

    Yes. We are looking to maximize performance, and I have over 15 years exper= ience with the languages.

    other languages suitable for server design, many with frameworks that
    =20
    will give you most of what you want here without any effort. Python
    =20
    with the "twisted" framework springs to mind (since I've used it myself)
    =20
    as being very efficient at handling multiple ports with multiple
    =20
    connections, and it takes care of all the details of resources.

    Yeah, I thought about Python. There are a few problems with it. (1) I can r= ead Python, but I don't write it very well (this is a big problem in practi=
    ce ;). (2) Python is interpreted and not native, so I think C/C++ will have=
    somewhat better performance. (3) Python makes some really dumb decisions o=
    ut of the box, and I don't know where all the devils lie (cf, this 2007 bug=
    : http://bugs.python.org/issue1589).

    Jeff

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From David Brown@1:0/0 to All on Wed Nov 27 09:56:10 2013
    On 26/11/13 18:47, noloader@gmail.com wrote:
    On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:
    <snip>>


    This is not really a programming group - but you might get some
    ideas

    here. I don't know what the ideal group would be - comp.lang.c
    and

    comp.lang.c++ are more about the language than applications.
    Oh, sorry about that.

    No apology needed - posting here is fine too. I'm just warning you that
    you won't get all the answers!

    However, if you are posting to a Usenet group, try to use a proper
    newsreader - Google Groups makes a hideous mess out of posts.


    I'll try another group or Stack Overflow.

    Also, is there a particular reason to pick C or C++ ?

    Yes. We are looking to maximize performance, and I have over 15 years experience with the languages.

    A lot of people think C and C++ are the answer to "maximum performance".
    Usually they are wrong.

    Once the shock of that wild generalisation has blown over, I'll explain
    my reasoning. C and C++ (along with other compiled languages like Ada, Fortran, ocaml, etc.) give you the smallest and fastest code at the
    detail level. But the speed of a system like this will depend on its structure, its threading, queuing, synchronisation, its use of blocking
    and non-blocking sockets, the algorithms used to track large data
    blocks, etc. These things are not dependent on the quality of a
    compiler or interpreter - they are dependent on the quality of the
    design. Higher level languages - such as Python - make it easier to
    make good designs than more detail-oriented lower level languages.
    Ergo, such code is often faster when written using a higher level language.

    Secondly, much of the work in a Python program is handled by the
    libraries, rather than being written by the developer. And much of that
    code, especially where there are /real/ performance gains to be made, is written in C. And since this code is written by people who have been
    willing to spend a lot of time and effort making efficient and
    well-tested code, such code is normally faster than you would write
    yourself (I don't know how good a programmer you are, but I assume you
    have other priorities such as development time and costs that limit your efforts in making the fastest possible code).

    So in theory, C and C++ should give the fastest server code. In
    practice, using the right existing framework greatly overwhelms any
    speed benefits from these languages.


    And assuming you are getting paid for your work here, you are /not/
    looking to maximize performance. You are looking for /good enough/
    performance - anything more is a waste of your employer's or client's
    money. That's worth keeping in mind, especially because employers and
    clients often /say/ they want "maximum performance". You are always
    looking for a balance between higher performance, and the costs of
    getting there.


    Of course, experience with the languages is another matter :-)



    other languages suitable for server design, many with frameworks
    that

    will give you most of what you want here without any effort.
    Python

    with the "twisted" framework springs to mind (since I've used it
    myself)

    as being very efficient at handling multiple ports with multiple

    connections, and it takes care of all the details of resources.

    Yeah, I thought about Python. There are a few problems with it.

    (1) I can read Python, but I don't write it very well (this is a big problem in practice ;).

    Python is a relatively easy language to work with (at least for simpler
    code), and your aim here is to use an existing framework for most of the system.

    If this is a big enough project, learning Python and writing the system
    in Python may be faster than writing it all in C++.

    (And Python is just an example - there are many others, such as Ruby on
    Rails.)


    (2) Python is interpreted and not native, so I think C/C++ will have
    somewhat better performance.

    Python is byte-compiled and runs in a virtual machine. Performance is
    good for some things, poor for others - it is very fast for manipulating
    data, handling strings and text, etc., but slow for basic arithmetic.
    And for performance you can always use PyPy, which has JIT compilation
    to give near-native performance.


    (3) Python makes some really dumb decisions out of the box, and I don't
    know where all the devils lie (cf, this 2007 bug: http://bugs.python.org/issue1589).


    That particular case is a "debatable" decision, rather than a "dumb"
    decision. And there will /always/ be such issues with whatever
    frameworks or libraries you choose to use - even if you write
    /everything/ yourself, you will feel some decisions in the Linux kernel
    are "dumb".

    But it is a good reason for picking well-known and well-tested languages
    and frameworks, rather than picking the latest and greatest trends.
    Python /is/ such a well-known and well-tested language, and Twisted is
    such a framework - they are massively popular.

    Just as a hint, Google is a major backer of Python development, and they
    use it for a lot of their servers and other software.



    I certainly don't mean to say that you should switch to Python for this project. But I do think that at this stage, you should keep your
    options open and look at the bigger picture, so that you can make an
    informed and justifiable decision.

    mvh.,

    David



    Jeff



    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From Aragorn@110:110/2002 to All on Wed Nov 27 11:14:31 2013
    On Wednesday 27 November 2013 10:56, David Brown conveyed the following
    to comp.os.linux.networking...

    On 26/11/13 18:47, noloader@gmail.com wrote:

    On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:

    This is not really a programming group - but you might get some
    ideas here. I don't know what the ideal group would be -
    comp.lang.c and comp.lang.c++ are more about the language than
    applications.

    Oh, sorry about that.

    No apology needed - posting here is fine too. I'm just warning you
    that you won't get all the answers!

    However, if you are posting to a Usenet group, try to use a proper
    newsreader - Google Groups makes a hideous mess out of posts.

    Not to mention that Google Groups is also one of the favorite posting
    media for spammers, which is why most of the regulars filter out
    anything coming in through Google Groups, which reduces a bona fide
    poster's chances of getting a useful reply.

    --
    = Aragorn =

    http://www.linuxcounter.net - registrant #223157

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: A noiseless patient Strider (110:110/2002@linuxnet)
  • From David Brown@1:0/0 to All on Wed Nov 27 12:58:17 2013
    On 27/11/13 12:14, Aragorn wrote:
    On Wednesday 27 November 2013 10:56, David Brown conveyed the following
    to comp.os.linux.networking...

    On 26/11/13 18:47, noloader@gmail.com wrote:

    On Tuesday, November 26, 2013 3:48:45 AM UTC-5, David Brown wrote:

    This is not really a programming group - but you might get some
    ideas here. I don't know what the ideal group would be -
    comp.lang.c and comp.lang.c++ are more about the language than
    applications.

    Oh, sorry about that.

    No apology needed - posting here is fine too. I'm just warning you
    that you won't get all the answers!

    However, if you are posting to a Usenet group, try to use a proper
    newsreader - Google Groups makes a hideous mess out of posts.

    Not to mention that Google Groups is also one of the favorite posting
    media for spammers, which is why most of the regulars filter out
    anything coming in through Google Groups, which reduces a bona fide
    poster's chances of getting a useful reply.


    Most of the spam I see on newsgroups is from direct posters - but I
    don't see much. Maybe my newsserver filters spam better than yours.

    But as long as there are people who filter out Google Groups posts, your
    point is well made.


    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From Jorgen Grahn@1:0/0 to All on Sun Dec 8 01:04:41 2013
    On Tue, 2013-11-26, David Brown wrote:
    On 26/11/13 07:39, noloader@gmail.com wrote:
    Hi All,

    Is anyone aware of a discussion or treatment of a multi-port threaded
    server on modern Linux boxes written in c/c++? Multi-port would mean,
    for example, ports 80 and 443.

    Contemporary Linux and Unix post-date W. Richard Steven's Advanced
    Programming in the Unix Environment and Unix Network Programming, so
    my bibles are not helpful. Googling is returning the Comp Sci 101
    examples with a single server/single port.

    Specifically, I'm interested in discussions of:
    ....

    Finally, my apologies if this is not a programming group.
    ....

    This is not really a programming group - but you might get some ideas
    here. I don't know what the ideal group would be - comp.lang.c and comp.lang.c++ are more about the language than applications.

    Yes they are.

    Comp.unix.programmer seems like the right place to me, but it has
    rather few posters (often with rather strong and unusual opinions).

    There is no such language as "C/C++". A server like this would be
    structured totally differently in C and in C++.

    I don't think so: unless you jump on the C++11 train, what determines
    the high-level structure is the Linux APIs you use, not if you use C
    or C++. At least in my experience. Of course C++ will let you do a
    lot of things safer and more elegantly, but not this part.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From Jorgen Grahn@1:0/0 to All on Sun Dec 8 01:25:15 2013
    On Tue, 2013-11-26, noloader@gmail.com wrote:

    Is anyone aware of a discussion or treatment of a multi-port
    threaded server on modern Linux boxes written in c/c++? Multi-port
    would mean, for example, ports 80 and 443.

    Contemporary Linux and Unix post-date W. Richard Steven's Advanced Programming in the Unix Environment and Unix Network Programming, so
    my bibles are not helpful.

    Sure about that? There have been updated editions of his books. I
    still use the ones from the 1990s, plus the man pages. epoll(7) is
    the only major newer interface I use ... on the other hand I don't do
    threads and maybe the state of the art has changed more there.

    Googling is returning the Comp Sci 101
    examples with a single server/single port.

    Yes; the quality of such search results is low. I still see new code
    being written by cargo-culting them; it's more primitive than what
    you'd get from Stevens' 20 years old edition.

    Specifically, I'm interested in discussions of:

    (1) multiple processes servicing a single port and thread the accept()
    (2) single process with multiple threads for each port and thread the
    accept()
    (3) the most efficient way to wait and handle connections
    (4) how to safely shutdown a multi-port, multi-threaded server

    I'm fairly certain fork/exec is no longer recommended, especially
    since critical libraries like OpenSSL are not fork-safe in a
    multi-threaded environment.

    Depending on what you want to do, fork is still the right choice ...

    I'd like to be able to answer your questions (or read the answers) but
    sorry, no. I'm playing with one or two projects of my own, but my
    design is based on a single process, epoll and non-blocking sockets.
    I expect it to be I/O-bound rather than CPU-bound, and I dislike and
    distrust threading ... so I don't think it's useful to you.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From Dave@110:110/2002 to All on Sun Dec 8 23:12:52 2013
    noloader@gmail.com wrote:


    Finally, my apologies if this is not a programming group. Google's Usenet interface simply sucks. We can't navigate group hierarchies any longer
    under the new and improved design. And searching for "comp.os.linux
    groups" returned one result of comp.os.linux.misc.

    The answer to that is to get a proper threaded newsreader and a subscription to an nntp server such as Eternal September, then you can experience proper newsgroups.

    Google's Usenet interface should be reserved for emergencies only.
    --
    Dave
    Too many gadgets, too little time

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: A noiseless patient Spider (110:110/2002@linuxnet)