Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
×
Open Source Supercomputing Networking Ruby

Peplum: F/OSS Distributed Parallel Computing and Supercomputing At Home With Ruby Infrastructure (ecsypno.com) 20

Slashdot reader Zapotek brings an update from the Ecsypno skunkworks, where they've been busy with R&D for distributed computing systems: Armed with Cuboid, Qmap was built, which tackled the handling of nmap in a distributed environment, with great results. Afterwards, an iterative clean-up process led to a template of sorts, for scheduling most applications in such environments.

With that, Peplum was born, which allows for OS applications, Ruby code and C/C++/Rust code (via Ruby extensions) to be distributed across machines and tackle the processing of neatly grouped objects.

In essence, Peplum:

- Is a distributed computing solution backed by Cuboid.
- Its basic function is to distribute workloads and deliver payloads across multiple machines and thus parallelize otherwise time consuming tasks.
- Allows you to combine several machines and built a cluster/supercomputer of sorts with great ease.

After that was dealt with, it was time to port Qmap over to Peplum for easier long-term maintenance, thus renamed Peplum::Nmap.

We have high hopes for Peplum as it basically means easy, simple and joyful cloud/clustering/super-computing at home, on-premise, anywhere really. Along with the capability to turn a lot of security oriented apps into super versions of themselves, it is quite the infrastructure.

Yes, this means there's a new solution if you're using multiple machines for "running simulations, to network mapping/security scans, to password cracking/recovery or just encoding your collection of music and video" -- or anything else: Peplum is a F/OSS (MIT licensed) project aimed at making clustering/super-computing affordable and accessible, by making it simple to setup a distributed parallel computing environment for abstract applications... TLDR: You no longer have to only imagine a Beowulf cluster of those, you can now easily build one yourself with Peplum.
Some technical specs: It is written in the Ruby programming language, thus coming with an entire ecosystem of libraries and the capability to run abstract Ruby code, execute external utilities, run OS commands, call C/C++/Rust routines and more...

Peplum is powered by Cuboid, a F/OSS (MIT licensed) abstract framework for distributed computing — both of them are funded by Ecsypno Single Member P.C., a new R&D and Consulting company.

This discussion has been archived. No new comments can be posted.

Peplum: F/OSS Distributed Parallel Computing and Supercomputing At Home With Ruby Infrastructure

Comments Filter:
  • Ruby? (Score:2, Informative)

    by Anonymous Coward

    And with ruby being so slow, you need a cluster to run something that would have been fine on a single machine if it were written in a more efficient language.

    • You beat me to it. Maybe they'll add Perl, too.

      • Seriously around me Ruby is gone... projects are cutting number of languages used and Ruby is one of the victims...

      • The business logic doesn't need bare metal performance, just the payload, and if that. You've kinda missed the entire objective of the project here.
        • You've kinda missed the entire objective of the project here.

          To get attention by wasting time? Mission accomplished!

          Every time I have to screw with something written in Ruby it's a headache just getting the tools going.

  • The very thought of this computing framework is going to keep me up at night at best. At worst it is nightmare fuel. Why did anyone ever bring this into the light of day?

    • Someone needed to have a grid for their Web application security scanner in order to load-balance scans but without system dependencies.
      Guilty btw, but why nightmare fuel?
  • ...but there's no way there aren't tons of other solutions for this already, for Python alone, that don't require you to learn how to compile C++ or Rust to include bindings into fecking Ruby.

  • AI has mostly gone the Python route. Cool languages went to Rust. K8s is mostly Go.

    I don't understand the market for this. Hadoop was usually for giant batch jobs. Mesos came and went.

    What's needed is a modern replacement for Rock Clusters: an integrated, manageable HPC Linux distro.

  • The standard solution in the actual supercomputing world is the venerable Message Passing Interface, of which the most accessible implementation is probably OpenMPI.

    Note that this distinct from OpenMP, which is a multi-cpu threading shared memory library.

    All of these solutions need to be written into the applications that use them, interchangeably with any other threading systems they may use. The quality of the integration varies greatly, and can heavily affect processing outcomes... but also the way they'

    • Right now this deals with divisible workloads. All there is for MPI are the Ruby RPC and a shared hashmap with callbacks to serve possibly even as a channel.
      I don't imagine anyone using this for other tasks although the project was just born and high-performance code can be added via native C/C++/Rust extensions to the project. That'd go into Cuboid though, not Peplum, Peplum would just benefit
      • yeah i see how the framework is set up, it does make sense to separate it out.

        As you say, the divisible workload is the key... but that makes me wonder why it's necessary in this particular implementation.On the other hand, if you're enjoying the work and someone finds it useful... well that's fine then.

        From my perspective there's an ongoing conversation around HPC and other types of distributed computing by people who don't know what the words computational complexity even mean... and so I'm used to having

  • ...a beowolf cluster of these

Chemist who falls in acid is absorbed in work.

Working...