• Re: New Python implementation

    From Chris M. Thomasson@chris.m.thomasson.1@gmail.com to comp.lang.c++ on Mon Feb 22 21:08:43 2021
    From Newsgroup: comp.lang.c++

    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    On Sat, 13 Feb 2021 12:52:50 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/13/2021 2:38 AM, mickspud@potatofield.co.uk wrote:
    On Fri, 12 Feb 2021 13:08:32 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/12/2021 8:20 AM, mickspud@potatofield.co.uk wrote:
    if you've only programmed on a toy OS like Windows that can even do
    something

    as fundamental as multiplexing network sockets without threading then I >> guess

    you may well be screwed without them.


    Are you familiar with IOCP?

    Nope, but having skim read about it I don't see anything there that multi >>> process couldn't accomplish. If you know otherwise then feel free to explain.



    IOCP is basically windows version of POSIX aio. Just make sure to never
    create a process and/or thread per connection. That really, really bad.

    I guess that depends on how long many connections you're expecting per unit time, how long they stay up and how important the server is. I wrote a middleware system that forked every time a new connection came but but we only
    expected a few connections a minute and they'd stay up for 2-3 mins at a time.
    Plus it was a critical system and if the main process went down the company backend would grind to a halt so multiplexing and threading were off the table.


    I have seen cases where the server created a thread-per-connection and
    all was well... Until more and more users started using it!
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From mickspud@mickspud@potatofield.co.uk to comp.lang.c++ on Tue Feb 23 09:18:36 2021
    From Newsgroup: comp.lang.c++

    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're expecting per unit >> time, how long they stay up and how important the server is. I wrote a
    middleware system that forked every time a new connection came but but we >only
    expected a few connections a minute and they'd stay up for 2-3 mins at a >time.
    Plus it was a critical system and if the main process went down the company >> backend would grind to a halt so multiplexing and threading were off the >table.


    I have seen cases where the server created a thread-per-connection and
    all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes
    it brings down the entire system. Thats really not something you want if its mission critical.

    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From Chris M. Thomasson@chris.m.thomasson.1@gmail.com to comp.lang.c++ on Tue Feb 23 01:26:22 2021
    From Newsgroup: comp.lang.c++

    On 2/23/2021 1:18 AM, mickspud@potatofield.co.uk wrote:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're expecting per unit >>> time, how long they stay up and how important the server is. I wrote a
    middleware system that forked every time a new connection came but but we >> only
    expected a few connections a minute and they'd stay up for 2-3 mins at a
    time.
    Plus it was a critical system and if the main process went down the company >>> backend would grind to a halt so multiplexing and threading were off the
    table.


    I have seen cases where the server created a thread-per-connection and
    all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes
    it brings down the entire system. Thats really not something you want if its mission critical.


    Your are correct... Therefore one should have multiple processes, and
    multiple threads within each process. Thank you for making me remember
    various setups I have coded:

    1 process per cpu

    multiple threaders per process for the cores in the cpu.


    Then you can get more coarse/fine grain:

    process per cpu, bound to it.

    core child process per core on target bound cpu, bound

    hyper threaded threads per-core child process.

    fibers in threads? Na...


    ect...


    Your are 100% correct that a single thread going down should NOT off the system/server. Your are right. Multiple processes needed.
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From Chris M. Thomasson@chris.m.thomasson.1@gmail.com to comp.lang.c++ on Tue Feb 23 01:28:06 2021
    From Newsgroup: comp.lang.c++

    On 2/23/2021 1:26 AM, Chris M. Thomasson wrote:
    On 2/23/2021 1:18 AM, mickspud@potatofield.co.uk wrote:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    [...]
    Your are 100% correct that a single thread going down should NOT off the system/server. Your are right. Multiple processes needed.


    Your means You are. Sorry.

    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From mickspud@mickspud@potatofield.co.uk to comp.lang.c++ on Tue Feb 23 11:50:25 2021
    From Newsgroup: comp.lang.c++

    On Tue, 23 Feb 2021 01:26:22 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/23/2021 1:18 AM, mickspud@potatofield.co.uk wrote:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    I have seen cases where the server created a thread-per-connection and
    all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes >> it brings down the entire system. Thats really not something you want if its >> mission critical.


    Your are correct... Therefore one should have multiple processes, and >multiple threads within each process. Thank you for making me remember >various setups I have coded:

    Yes, thats certainly an approach I've seen a lot. Of course marshalling both processes and threads can be a difficult task and programmers who are well versed in both multithreaded and multiprocess development are not as common as they should be.

    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From scott@scott@slp53.sl.home (Scott Lurndal) to comp.lang.c++ on Tue Feb 23 14:59:04 2021
    From Newsgroup: comp.lang.c++

    mickspud@potatofield.co.uk writes:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're expecting per unit >>> time, how long they stay up and how important the server is. I wrote a
    middleware system that forked every time a new connection came but but we >>only
    expected a few connections a minute and they'd stay up for 2-3 mins at a >>time.
    Plus it was a critical system and if the main process went down the company >>> backend would grind to a halt so multiplexing and threading were off the >>table.


    I have seen cases where the server created a thread-per-connection and
    all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes
    it brings down the entire system. Thats really not something you want if its >mission critical.


    Your premise is faulty.
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From mickspud@mickspud@potatofield.co.uk to comp.lang.c++ on Tue Feb 23 15:19:22 2021
    From Newsgroup: comp.lang.c++

    On Tue, 23 Feb 2021 14:59:04 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:
    mickspud@potatofield.co.uk writes:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're expecting per unit

    time, how long they stay up and how important the server is. I wrote a >>>> middleware system that forked every time a new connection came but but we >>>only
    expected a few connections a minute and they'd stay up for 2-3 mins at a >>>time.
    Plus it was a critical system and if the main process went down the company

    backend would grind to a halt so multiplexing and threading were off the >>>table.


    I have seen cases where the server created a thread-per-connection and >>>all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes >>it brings down the entire system. Thats really not something you want if its >>mission critical.


    Your premise is faulty.

    No, it isn't.

    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From scott@scott@slp53.sl.home (Scott Lurndal) to comp.lang.c++ on Tue Feb 23 16:20:27 2021
    From Newsgroup: comp.lang.c++

    mickspud@potatofield.co.uk writes:
    On Tue, 23 Feb 2021 14:59:04 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:
    mickspud@potatofield.co.uk writes:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're expecting per unit

    time, how long they stay up and how important the server is. I wrote a >>>>> middleware system that forked every time a new connection came but but we >>>>only
    expected a few connections a minute and they'd stay up for 2-3 mins at a >>>>time.
    Plus it was a critical system and if the main process went down the company

    backend would grind to a halt so multiplexing and threading were off the >>>>table.


    I have seen cases where the server created a thread-per-connection and >>>>all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes >>>it brings down the entire system. Thats really not something you want if its >>>mission critical.


    Your premise is faulty.

    No, it isn't.

    Any modern operating system is multithreaded. They recover just fine
    when "1 thread crashes". Ipso facto.
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From mickspud@mickspud@potatofield.co.uk to comp.lang.c++ on Tue Feb 23 16:44:37 2021
    From Newsgroup: comp.lang.c++

    On Tue, 23 Feb 2021 16:20:27 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:
    mickspud@potatofield.co.uk writes:
    On Tue, 23 Feb 2021 14:59:04 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:
    mickspud@potatofield.co.uk writes:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're expecting per >unit

    time, how long they stay up and how important the server is. I wrote a >>>>>> middleware system that forked every time a new connection came but but we

    only
    expected a few connections a minute and they'd stay up for 2-3 mins at a >>>>>time.
    Plus it was a critical system and if the main process went down the >company

    backend would grind to a halt so multiplexing and threading were off the >>>>>table.


    I have seen cases where the server created a thread-per-connection and >>>>>all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes >>>>it brings down the entire system. Thats really not something you want if its

    mission critical.


    Your premise is faulty.

    No, it isn't.

    Any modern operating system is multithreaded. They recover just fine
    when "1 thread crashes". Ipso facto.

    If you're talking about the OS kernel, no, they don't always recover just fine if something goes wrong. Its called a Kernel panic or blue screen in Windows. However the OS controls what happens with itself, user processes have limited control over what the OS does to them when they page fault or similar.

    HTH.

    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From scott@scott@slp53.sl.home (Scott Lurndal) to comp.lang.c++ on Tue Feb 23 17:34:00 2021
    From Newsgroup: comp.lang.c++

    mickspud@potatofield.co.uk writes:
    On Tue, 23 Feb 2021 16:20:27 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:

    Your premise is faulty.

    No, it isn't.

    Any modern operating system is multithreaded. They recover just fine
    when "1 thread crashes". Ipso facto.

    If you're talking about the OS kernel, no, they don't always recover just fine >if something goes wrong. Its called a Kernel panic or blue screen in Windows.

    Windows is hardly an exemplar of operating system design. I've been
    writing operating systems for four decades, and I stand by my assertion.

    However the OS controls what happens with itself, user processes have limited >control over what the OS does to them when they page fault or similar.

    It is quite possible for applications to be quite resiliant, when using
    real operating systems. Tandem non-stop comes to mind, and they're still chugging away (HPE).
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From Chris M. Thomasson@chris.m.thomasson.1@gmail.com to comp.lang.c++ on Tue Feb 23 11:25:51 2021
    From Newsgroup: comp.lang.c++

    On 2/23/2021 8:20 AM, Scott Lurndal wrote:
    mickspud@potatofield.co.uk writes:
    On Tue, 23 Feb 2021 14:59:04 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:
    mickspud@potatofield.co.uk writes:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're expecting per unit

    time, how long they stay up and how important the server is. I wrote a >>>>>> middleware system that forked every time a new connection came but but we
    only
    expected a few connections a minute and they'd stay up for 2-3 mins at a >>>>> time.
    Plus it was a critical system and if the main process went down the company

    backend would grind to a halt so multiplexing and threading were off the >>>>> table.


    I have seen cases where the server created a thread-per-connection and >>>>> all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes >>>> it brings down the entire system. Thats really not something you want if its
    mission critical.


    Your premise is faulty.

    No, it isn't.

    Any modern operating system is multithreaded. They recover just fine
    when "1 thread crashes". Ipso facto.


    In user space, when a thread in a process crashes, it brings the whole
    process down.
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From Chris M. Thomasson@chris.m.thomasson.1@gmail.com to comp.lang.c++ on Tue Feb 23 11:26:38 2021
    From Newsgroup: comp.lang.c++

    On 2/23/2021 11:25 AM, Chris M. Thomasson wrote:
    On 2/23/2021 8:20 AM, Scott Lurndal wrote:
    mickspud@potatofield.co.uk writes:
    On Tue, 23 Feb 2021 14:59:04 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:
    mickspud@potatofield.co.uk writes:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/15/2021 1:34 AM, mickspud@potatofield.co.uk wrote:
    I guess that depends on how long many connections you're
    expecting per unit

    time, how long they stay up and how important the server is. I
    wrote a
    middleware system that forked every time a new connection came
    but but we
    only
    expected a few connections a minute and they'd stay up for 2-3
    mins at a
    time.
    Plus it was a critical system and if the main process went down >>>>>>> the company

    backend would grind to a halt so multiplexing and threading were >>>>>>> off the
    table.


    I have seen cases where the server created a thread-per-connection >>>>>> and
    all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread
    crashes
    it brings down the entire system. Thats really not something you
    want if its
    mission critical.


    Your premise is faulty.

    No, it isn't.

    Any modern operating system is multithreaded.  They recover just fine
    when "1 thread crashes".  Ipso facto.


    In user space, when a thread in a process crashes, it brings the whole process down.

    Well, that has been my experience.
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From Chris M. Thomasson@chris.m.thomasson.1@gmail.com to comp.lang.c++ on Tue Feb 23 11:28:38 2021
    From Newsgroup: comp.lang.c++

    On 2/23/2021 3:50 AM, mickspud@potatofield.co.uk wrote:
    On Tue, 23 Feb 2021 01:26:22 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    On 2/23/2021 1:18 AM, mickspud@potatofield.co.uk wrote:
    On Mon, 22 Feb 2021 21:08:43 -0800
    "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> wrote:
    I have seen cases where the server created a thread-per-connection and >>>> all was well... Until more and more users started using it!

    The big achilles heal of multithreaded systems is that if 1 thread crashes >>> it brings down the entire system. Thats really not something you want if its
    mission critical.


    Your are correct... Therefore one should have multiple processes, and
    multiple threads within each process. Thank you for making me remember
    various setups I have coded:

    Yes, thats certainly an approach I've seen a lot. Of course marshalling both processes and threads can be a difficult task and programmers who are well versed in both multithreaded and multiprocess development are not as common as
    they should be.


    Indeed! This brings to mind so-called robust mutexes. I have used to use
    them to detect when a process dies while holding something critical
    shared between processes. Windows and POSIX has them.
    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From mickspud@mickspud@potatofield.co.uk to comp.lang.c++ on Wed Feb 24 09:23:20 2021
    From Newsgroup: comp.lang.c++

    On Tue, 23 Feb 2021 17:34:00 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:
    mickspud@potatofield.co.uk writes:
    On Tue, 23 Feb 2021 16:20:27 GMT
    scott@slp53.sl.home (Scott Lurndal) wrote:

    Your premise is faulty.

    No, it isn't.

    Any modern operating system is multithreaded. They recover just fine >>>when "1 thread crashes". Ipso facto.

    If you're talking about the OS kernel, no, they don't always recover just fine

    if something goes wrong. Its called a Kernel panic or blue screen in Windows.

    Windows is hardly an exemplar of operating system design. I've been
    writing operating systems for four decades, and I stand by my assertion.

    I was thinking of *nix mainly, my Windows dev experience is limited.

    However the OS controls what happens with itself, user processes have limited >>control over what the OS does to them when they page fault or similar.

    It is quite possible for applications to be quite resiliant, when using
    real operating systems. Tandem non-stop comes to mind, and they're still >chugging away (HPE).

    Tandem was a specialist OS and will probably diverge from common OS's in how threads and processes are demarcated. Anyway, the non-stop part is referring to the OS maintaining the processes if some hardware goes down, not keeping them alive if the process itself crashes.

    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From Alf P. Steinbach@alf.p.steinbach+usenet@gmail.com to comp.lang.c++ on Wed Feb 24 17:16:59 2021
    From Newsgroup: comp.lang.c++

    On 11.02.2021 13:29, Mr Flibble wrote:


    I am starting work on creating a new Python implementation from scratch
    [snip]

    Possibly relevant: the Julia programming language, e.g. see https://www.nature.com/articles/d41586-019-02310-3

    - Alf

    --- Synchronet 3.18c-Linux NewsLink 1.113
  • From mickspud@mickspud@potatofield.co.uk to comp.lang.c++ on Wed Feb 24 17:01:47 2021
    From Newsgroup: comp.lang.c++

    On Wed, 24 Feb 2021 17:16:59 +0100
    "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com> wrote:
    On 11.02.2021 13:29, Mr Flibble wrote:


    I am starting work on creating a new Python implementation from scratch >[snip]

    I missed that recent bit of bullshittery from the groups resident comedian. There needs to be a new category beyond vapourware for his software,
    perhaps Heisenbergware to go along with Heisenbugs - when you look for it it magically vanishes.

    --- Synchronet 3.18c-Linux NewsLink 1.113