Hive


Executed testsuites

These testsuites are available, and can be loaded. Click on 'Load' to load a certain suite.

If you load several suites, you can compare individual testcases on different runs/suites/clients

Execution results: Nothing loaded yet

Nothing loaded yet

About Hive

Before Hive, we did have consensus-tests, in mainly two forms -- statetests and blockchain tests (yeah there are a few more). The tests were generated via aleth/testeth, and put on a repository. From that repo, it was expected that client teams import the tests and execute within their own custom test-harnesses.

Now, that didn't work out perfectly. Client teams didn't always pull in the latest tests, and whether they actually executed the tests as intended was not sure. Did they correctly verify everything? It was especially troublesome when hardforks were approaching, and "oops, hey, we commented out test XX because it was failing".

Then Péter Szilágyi (@karalabe) had this idea, to run clients totally black box. You spin up a node inside a docker container, start it with the client-specific config, and feed it the blocks. Another docker container 'orchestrates' the test, and downloads the entire test repo from github (blockchain tests contain the RLP-blocks). So, after asking the client to import the blocks, all the 'orchestrator' needs to do is query "what's your latest block?", and verify it aginst the expected lastblock from the testcase.

Those tests are what one normally talks about when talking about "hive-tests". However, Hive is even more versatile, since another "orchestrator" can do other things -- they can send any network message they want to the node-under-test. So far, we also have tests that verify that fast-sync can be performed, and tests to check that the p2p implementation does not make it possible to do DoS via amplification attacks. So those are also hive-tests.

However, an important distinction: Hive is, in essence, a tool to

  • execute tests
  • in an automated way,
  • treating the clients as black boxes as far as possible.

About the EF Hive executor

For the last ~3 years, we have had a hive-server running in production. https://hivetests.ethdevops.io/. It runs 24/7, and always fetches the latest tests from ethereum/tests repo.


Debug log