Crate polkadot_node_core_pvf

A crate that implements the PVF validation host.

For more background, refer to the Implementer’s Guide: PVF Pre-checking and Candidate Validation.

Entrypoint

This crate provides a simple API. You first start the validation host, which gives you the handle and the future you need to poll.

Then using the handle the client can send three types of requests:

(a) PVF pre-checking. This takes the PVF code and tries to prepare it (verify and compile) in order to pre-check its validity.

(b) PVF execution. This accepts the PVF params and the PVF code, prepares (verifies and compiles) the code, and then executes PVF with the params.

(c) Heads up. This request allows to signal that the given PVF may be needed soon and that it should be prepared for execution.

The preparation results are cached for some time after they either used or was signaled in heads up. All requests that depends on preparation of the same PVF are bundled together and will be executed as soon as the artifact is prepared.

Priority

PVF execution requests can specify the priority with which the given request should be handled. Different priority levels have different effects. This is discussed below.

Preparation started by a heads up signal always starts with the background priority. If there is already a request for that PVF preparation under way the priority is inherited. If after heads up, a new PVF execution request comes in with a higher priority, then the original task’s priority will be adjusted to match the new one if it’s larger.

Priority can never go down, only up.

Under the hood

The flow

Under the hood, the validation host is built using a bunch of communicating processes, not dissimilar to actors. Each of such “processes” is a future task that contains an event loop that processes incoming messages, potentially delegating sub-tasks to other “processes”.

Two of these processes are queues. The first one is for preparation jobs and the second one is for execution. Both of the queues are backed by separate pools of workers of different kind.

Preparation workers handle preparation requests by prevalidating and instrumenting PVF wasm code, and then passing it into the compiler, to prepare the artifact.

Artifacts

An artifact is the final product of preparation. If the preparation succeeded, then the artifact will contain the compiled code usable for quick execution by a worker later on.

If the preparation failed, then the worker will still write the artifact with the error message. We save the artifact with the error so that we don’t try to prepare the artifacts that are broken repeatedly.

The artifact is saved on disk and is also tracked by an in memory table. This in memory table doesn’t contain the artifact contents though, only a flag that the given artifact is compiled.

A pruning task will run at a fixed interval of time. This task will remove all artifacts that weren’t used or received a heads up signal for a while.

Execution

The execute workers will be fed by the requests from the execution queue, which is basically a combination of a path to the compiled artifact and the params.

Re-exports

• pub use sc_executor_common;
• pub use sp_maybe_compressed_blob;

Macros

• decl_puppet_worker_main
  Use this macro to declare a fn main() {} that will check the arguments and dispatch them to the appropriate worker, making the executable that can be used for spawning workers.

Structs

• Config
  Configuration for the validation host.
• Metrics
  Validation host metrics.
• Pvf
  A struct that carries code of a parachain validation function and its hash.
• ValidationHost
  A handle to the async process serving the validation host requests.

Enums

• InvalidCandidate
  A description of an error raised during executing a PVF and can be attributed to the combination of the candidate polkadot_parachain::primitives::ValidationParams and the PVF.
• PrepareError
  An error that occurred during the prepare part of the PVF pipeline.
• Priority
  A priority assigned to execution of a PVF.
• ValidationError
  A error raised during validation of the candidate.

Constants

• JOB_TIMEOUT_WALL_CLOCK_FACTOR
  A multiple of the job timeout (in CPU time) for which we are willing to wait on the host (in wall clock time). This is lenient because CPU time may go slower than wall clock time.

Functions

• execute_worker_entrypoint
  The entrypoint that the spawned execute worker should start with. The socket_path specifies the path to the socket used to communicate with the host.
• prepare
  Runs preparation on the given runtime blob. If successful, it returns a serialized compiled artifact which can then be used to pass into Executor::execute after writing it to the disk.
• prepare_worker_entrypoint
  The entrypoint that the spawned prepare worker should start with. The socket_path specifies the path to the socket used to communicate with the host.
• prevalidate
  Runs the prevalidation on the given code. Returns a RuntimeBlob if it succeeds.
• start
  Start the validation host.

Type Definitions

• PrepareResult
  Result of PVF preparation performed by the validation host. Contains the elapsed CPU time if successful