Deviota Hercules: flexing the muscles

This is a small update on Hercules as requested by the community. We had a fair amount of progress in the past few weeks even though our main project is taking most of our time.

There are four areas we are working on that we want to have ready before a public preview (apart of cleaning up the code and writing the documentation, d’oh!).

Core

Above all, the core has to be stable. This includes the tangle database management, communication between nodes, confirmation of transactions, tip selection, milestones detection and all the related stuff.

Most of our time has been vested here in order to achieve feature parity with IRI. It is basically done. Our current tests show that Hercules is about 10–25% faster than IRI, uses about 10% less drive space and A LOT less memory under certain conditions.

Apart of that, the database structure is a little different from IRI, which allows certain things not possible before. For example, you do not have to reattach addresses after a snapshot.

Speaking of the devil:

Snapshots

Currently this is what we are working on. Hercules can successfully import IRI snapshots, load and save it’s own snapshots at a given time in the past and check it’s integrity.

Right now I am working on transactions trimming that takes a little more time due to peculiar issue we are facing when bulk-deleting items from thebadger database. However, I am confident that this issue is solvable within a reasonable timeframe.

The next step is to add automatic snapshots (cron jobs), smart snapshots (based on memory available) and even an API to trigger a snapshot, list and download available snapshots.

The snapshot API can be made public so that external tools, services and nodes can use it to compare snapshots from multiple nodes (if those have the same timestamp), generate tangle statistics, track address balances, speed up node sync and a myriad of other use cases. This should not be underestimated.

API

The API itself, comparing with snapshots and core, is simple and can be done within 4–6 days. It will mimic most of the IRI’s commands and behaviour, adding a few extra things like the snapshots API.

Given the different database structure, the API responses should be faster and take less resources than IRI. This is pure gut feeling, because there was no way to test that, yet.

Since our focus has been on core and snapshots, there is no usable API at the moment apart of “get node info” and neighbour management commands. This is also the main reason preventing us from pre-releasing Hercules, since it has no usage for anyone without a powerful API.

 

Support for low-end devices

This is the killer feature we promised for Hercules and we are putting a lot of core work to optimise memory usage and performance as far as it can be done. I hope that when Hercules is out, the community’ go-lang gurus can give us a hand to optimise it even further.

I started the latest Hercules version on PI3 today. It synchronises nicely and processes up to 100–120 transactions per second. This is up from 30–60 max transactions when I first tried running Hercules.

If the network speed grows, there will be no other way to completely sync a sub-node without a fairly recent snapshot. This is why the snapshots are a vital part of Hercules and its usage in tough environments.

Today I managed to run Hercules on a single-core 512MB Beaglebone Black doing cute 15–30 transactions per second. Nothing world-changing, but is probably usable if snapshots provided.

Single core. 512MB. Let that sink in. How far can we go? The main specs are comparable to those of a Pi Zero. However Pi Zero has an even slower memory. Really excited to see if we can get a decent speed on that one, too!

Let’s see if we can manage run run Hercules on a Casio watch anytime soon. 🙂

ETA

No ETA given our current workload. If everything goes well, it might be ready for public alpha within a few weeks. But don’t take my word on that.

Thanks for reading and for your support!

Roman Semko

IOTA Donations welcome:

NOT_ANYMORE_THANK_YOU
2018-08-03T11:31:26+00:00