Fix some errors with the storage report. I realized that the off_t type (in the struct pv_storage) is normally defined as a signed integer on POSIX and in another hand, the types inside of the struct statfs involved in the storage size calculation are __fsword_t that type is for internal use and the recommendation is to use an unsigned int and fsblkcnt_t who is defined as "unsigned integer type" so I change those off_t to unsigned to avoid the overflow. Also finally I change the pv_json_ser_int by pv_json_ser_number because the underlying library uses double

1. we will move to explicit .init_done marker file that we only place if whole do_crypt_init function has succeeded with changes synched to disk
2. we only do losetup logic for dm_type caam
3. we create a .goodbak copy of the image file every time we successfully finish with do_mount_disk; we are extra careful and create a .prebak copy of the image file before the mounting happens.
4. we make do_mount_disk try the following before giving up: - mount crypt disk as usual - if fails run an fsk.ext4 -y on that /dev/mapper/ device - if that fails we put back the .goodbak file (if such exist) at code we do this through recursive invocation of the do_mount_disk function with a 4th parameter 'specialrun' mode

A bunch of small changes I did while testing appengine shuts down before WAIT state is reached. List of changes:

• ctrl: unlink before binding to remove existing pv-ctrl sockets
• logserver: merge close and stop functions
• logserver: add fd closing before stopping the PID, which was causing a freeze
• pantavisor: stop logserver after we stop ph_logger, platforms and pvctrl, but before unmount and free up memory
• ph_logger: improve logs when stopping services

With this PR, we enable any string in run.json "runlevel", making it equivalent to "group". In case the two of them are configured and their values differ, the revision will be set as WONTGO.

Some change was added to this PR:

• Add better function to unescape chars and replace the old pv_str_unescape_to_ascii from utils/str.h
• Change the strategy of how user-meta is unescaped, now the data is unescaped after parsing

Both changes help to fix a nasty bug I found if you create a variable with any name but with the value ooo\nooo (something with a \n in between) the device will freeze completely.

crypt failures with CAAM are sporadic and no meaningful debug information available with existing state. Adapt volume handler and add additional debug logs in error exit path.

This PR refactors the logging system. Creates the logserver who centralizes the log processing allowing two different log modes:

• singlefile: in this mode, all logs (syslog, pantavisor logs, containers logs, lxc console, etc) are centralized in one file called pv.log that file is located in the folder of the current revision, inside of the logs/ directory
• filetree: this is the traditional log organization

Both modes are not exclusives and can be set at the same time. Important: currently only is possible to send logs to the cloud if the filetree option is not set.

To set the different log modes, a new configuration option called log.server.outputs was added to the pantavisor.config file, the different modes can be set using a comma as a separator. For example:

log.server.outputs=singlefile,filetree

All current log options are working and the only issues/missing features are:

• Currently the logs from the ph_logger fork are not sent to the log
• The logs from logs/current/<container>/lxc/lxc.log aren't capture in singlefile mode

dm-crypt: add pv_disk with dm-crypt

add initial support for dm-crypt based pv_disks. current implementation handles only CAAM and DCP explicit.

disk defaults to persistent storage. Brief features list are,

• disks.json: added new disks.json for handling the control over physical meidum to use. Refer 1 for more details.
• disk types: There are 4 disk types introduced. - Directory (default): will be used to mount to pvroot as default - dm-crypt-versatile: use the cryptsetup utils to mount the disk/file encrypted with plain text key - dm-crypt-dcp: specific to i.Mx2 & i.Mx6 low spec variants with hardware DCP module. Compared with versatile, the key will be encrypted using Hardware DCP and cryptsetup uses encrypted key blob - dm-crypt-caam: specific to all modern i.Mx platforms with CAAM. Encryption of the key and cryptsetup uses the black blob
• run.json: to consume the disks to use, defined the disk entry to run.json. Refere 2 for more details.

Signed-off-by: Parthiban Nallathambi [email protected]

Implements groups timeout:

• rollback if a group cannot achieve his status-goal before the timeout
• unblock platform boot if previous group cannot achieve his status-goal

decrypt the with dcp-tool decrypt and consume as plain text with cryptsetup plain.

Signed-off-by: Parthiban Nallathambi [email protected]

Currently, in non-reboot transitions, Pantavisor keeps the current state running for the new revision, but transfers the new/updated platforms from the pending state to the running one.This does not affect groups, as they are not changed whatsoever during non-reboot updates. We still have to re-link the platform reference list from the pending state to the current one.

Before this fix, Pantavisor was just re-linking the platforms belonging to the new revision, but the old ones belonging to platforms that were deleted or platforms that were modified were still there. This was causing Pantavisor to keep groups with lists of platform references that pointed to deleted structs etc.

With this fix, the list of references for each group are deleted during transitions, so the platforms belonging to the new revision can be cleanly re-linked to the groups.

Pantavisor can now load different policies at boot time. A policy is a configuration file following the same format as pantavisor.config. The files are located in /etc/pantavisor/policies/, so they can be added to each platform from vendor/stepskel/etc/pantavisor/policies/. The policy is set in the first stages of configuration (pantavisor.config or cmdline) and then loaded before the latest stages (initrdconfig or metadata config).

For example, if we wanted to load this policy from vendor:

vendor/stepskel/etc/pantavisor/policies/prod.config

We would have to set this key in cmdline:

pv_policy=prod

Another thing that would work is to set it in pantavisor.config

policy=prod

List of changes:

• config: add new "policy" key to pantavisor.config, overridable with pv_ suffix in cmdline
• config: new "policy" key is a string without '/' characters
• path: add new dynamic path for policies

This PR moves the loading of pantahub.config out of the early init so we can decide whether to load it or not based on Pantavisor being in remote or local mode. The idea is to only load the config file after bootup or transition once. Then, it is saved only after registration.

List of changes:

• config: stop supporting deprecated pantahub.config keys
• config: remove pantahub.config load from early init
• config: improve logs for load/save pantahub.config
• pantahub: remove loading of pantahub.config
• pantavisor: load pantahub.config in state RUN only